I sort a query result by using:
ORDER BY STR_TO_DATE(table1.date, '%Y%m%d'); -- the format of `date` is `Ymd` without delimiter.
However, the result is undesired:
20150508
20150514
20150525
20150514
20150515
20150522
20150525
20150529
20150605
20150612
20150619
Sorry for my carefulness. I use the shell command sort (./batch_queries.sh | cut -f 3-9,12- | sort | uniq -c) to sort the query result, leading to such a undsired result.
Related
I'm having an issue with a HAVING clause on a string that is created by a GROUP_CONCAT statement.
Running the query:
SELECT row_id, GROUP_CONCAT(cell_id,'-',cell_type) AS flat_row FROM `cells`
GROUP by row_id;
returns the following:
|--------|-------------------------------------------------|
| row_id | flat_row |
|--------|-------------------------------------------------|
| 1 | 1-Text,6-Text,45-Text,5-Number,37-Text,9-Number |
However, running
SELECT row_id, GROUP_CONCAT(cell_id,'-',cell_type) AS flat_row FROM `cells`
GROUP by row_id
HAVING flat_row = '1-Text,6-Text,45-Text,5-Number,37-Text,9-Number';
returns 0 records.
I was expecting to see the same result, does anyone know why the result is empty?
The goal here is to check whether the database contains rows that have the exact cell composition provided in the HAVING clause.
The issue has been resolved. It was a combination of making sure that the order of the string was always the same on both sides of the comparison and properly escaping backslashes in the expected data. Thanks to Caius Jard and Bill Karwin for pointing me in the right direction!
I am trying to query the total count of a partition in BigQuery and store the result in a mysql table. I am doing this for monitoring purpose.
#!/bin/sh
query1="bq query --format=csv 'SELECT COUNT(1) as Count FROM [dataset.tablename] WHERE _PARTITIONTIME = TIMESTAMP(\"$date\")'"
eval result=\$\($query1\)
echo "$result"
bq_insertquery="insert into <<table>>(<<column>>) values(${result})"
echo $bq_insertquery | mysql -u root -p <<dbname>>
Am getting error while insertion in mysql table. This is probably because the variable $result includes both the header and the value, i.e.
Variable $result holds: value with the header
Looks like myquery will be able to insert data, if i get only the value.
How should i assign only value to a shell variable, and use it thereafter ?
Edit: Any sql output contains column_name and values. The variable i assigned to store the value from BigQuery also contains both, i.e. column_name and value. I am looking for something which would be helpfull in extracting only value.
Simply add the --quiet flag (ignore status updates while jobs are running), and pipe it to awk:
query1="bq query --quiet --format=csv 'SELECT COUNT(1) as Count FROM [dataset.tablename] WHERE _PARTITIONTIME = TIMESTAMP(\"$date\")' | awk '{if(NR>1)print}'"
I would like to use jq command to parse the json output from the query. Before that you need to install jq command first. Here is the way to put count into a
result=$(echo -e "select 1 as col" | bq query --nouse_legacy_sql --format=json)
echo $result
it shows the output [{"col":"1"}]. Now it's time to use jq command to get final output.
count=$(echo $count | jq '.[0]' | jq '.col')
echo $count
In One line:
count=$(echo $(echo -e "select 1 as col" | bq query --nouse_legacy_sql --format=json) | jq '.[0]' | jq '.col')
I have a table with following columns and sample data
File Name | Status
'xxx_2015-07-20.csv' | Completed
'xxx_2015-07-19.csv' | Completed
'xxx_2015-07-18.csv' | Failed
.
.
'xxx_2015-06-01.csv' | Failed
Now I have two scenarios in my application (PHP-MySQL):
1) I have to fetch the status of today's file. I can do it like this:
select status from myTable where file_name like '%date(Y-m-d)%';
2) I want to fetch the status of all files generated since 1 month from today. Suppose today is 2015-07-20, then files from 2015-06-20 should show up.
Since there is no date column, I can't apply comparison operator and get the results. I believe it will need some playing with regular expressions.
I am not familiar with regular expressions, so any help on this would be much appreciated.
If the pattern is same i.e. xxx_2015-07-20.csv you can use substring_index function to get the date value as
mysql> select substring_index(substring_index('xxx_2015-07-20.csv','_',-1),'.',1) as d;
+------------+
| d |
+------------+
| 2015-07-20 |
+------------+
Now using the same you can have the select statement as
select status from myTable
where
substring_index(
substring_index(file_name,'_',-1),
'.',1
) = curdate();
Just wondering if there's a command line argument in mysql to remove the lines surrounding a mysql query result.
For one of my recent queries, I got this back:
+----------+
| count(*) |
+----------+
| 1016442 |
+----------+
Since I'd like to use this result in my shell script to do something else with it, I'd like it to just return the value, and not the +-----+ and | characters surrounding it. Is it possible to do this or do I have to find a way to parse around it? Thanks in advance for your help.
EDIT: I was hoping there was a mysql command line option to easily return just the result. If not, then I'll use sed, awk, grep like someone mentioned in the comments =)
use -s for what you want.
$ mysql -s
mysql> SELECT NOW();
NOW()
2014-04-25 10:11:57
-s means silent
$ mysql --help
...
-s, --silent Be more silent. Print results with a tab as separator,
Also you can skip column header with -N
$ mysql -s -N
mysql> SELECT NOW();
2014-04-25 10:14:49
Mysql by default prints table results in mysql table formatting
+----+----------+-------------+
| id | name | is_override |
+----+----------+-------------+
| 1 | Combined | 0 |
| 2 | Standard | 0 |
+----+----------+-------------+
When calling mysql from the unix shell this table formatting is not preserved, but it's easy to request it via the -t option
mysql -t my_schema < my_query_file.sql
When using Ruby, I'm using the mysql gem to return results. Since the gem returns data as hashes, there's no option to preserve table formatting. However, is there any way I can easily print a hash/data with that formatting? Without having to calculate spacing and such?
db = Mysql.new(my_database, my_username, my_password, my_schema)
result = db.query("select * from my_table")
result.each_hash { |h|
# Print row. Any way to print it with formatting here?
puts h
}
Some gems and codes:
https://rubygems.org/gems/datagrid
http://rubygems.org/gems/text-table
https://github.com/visionmedia/terminal-table
https://github.com/geemus/formatador
https://github.com/wbailey/command_line_reporter
https://github.com/arches/table_print
http://johnallen.us/?p=347
I have not tried any of them.