I have a text column which contains hashtags used by users. each row contains a different number of hashtags, separed by a space, like this:
USERS | HASHTAG COLUMN:
------------------------
user1 | hashtag1 hashtag2
user2 | hashtag2
user1 | hashtag1 hashtag2 hashtag3 hashtag4
I want to get the most used hashtags, in this case my desired output should be:
OCCURRENCES | TAG
----------------------
3 | hashtag2
2 | hashtag1
1 | hashtag3
1 | hashtag4
I have NO IDEA how to get it, any help is much appreciated. Thank you
Assuming you can't redesign your database to be in 1NF, then you can do this in bash:
echo "select hashtag from table" | \
mysql --user=foo --password=bar --host=hostname --database=dbname --skip-column-names | \
sed -e 's/ /\n/g' | \
sort | \
uniq -c | \
sort -rn
The sed command puts each hashtag on its own line. The first sort command puts all the duplicate hashtags next to each other so that ... the uniq command can count all the occurrences of each one. The second sort command orders the output in reverse numerical order by the counts.
Related
I am having issues and not understanding what the underlying cause is.
I have a table that has these three fields:
______________________________________________
| cid | order_id | TxRefNum |
----------------------------------------------
I am making a simple call in my bash script (there is literally no other code to start with)
#!/bin/bash
mysql --login-path=main-data -e "SELECT
cid,
order_id,
TxRefNum
FROM database.orders_temp" |
while read this that other; do
echo "$this || $that || $other"
done
I would expect to see the following:
__________________________________________________________
| 29 | F0VIc - CHATEAU ROOFIN | 5555555 |
----------------------------------------------------------
Instead my script is splitting the string $that into two different strings .. The echo is actually:
___________________________________________________
| 29 | F0VIc | - CHATEAU ROOFIN |
---------------------------------------------------
Do I have to set a delimiter when setting my variables in my while loop?? I am truly stumped!!
Getting output from the mysql command formatted in an intelligent way is problematic. In your case bash is interpreting the as a delimiter. You need to split a different way. I was able to get this working. You'll note the | in the query as well at the IFS line at the tope
#!/bin/bash
IFS='|' # set the delimiter
mysql --login-path=main-data -e "SELECT
29 as cid, '|',
'F0VIc - CHATEAU ROOFIN' as order_id,
'|',
5555555 as TxRefNum
FROM dual" |
while read this that other; do
echo "$this || $that || $other"
done
at first, I am a very new on shell scripting, so please don't shoot me !! :)
What I try to do. I have a multi-site WordPress installation, and I like to write a script that will be able to export specific tables from the schema either by passing the site id as argument in shell script, or by set an option to export all selected the tables of the schema.
The WordPress, in order to recognize which table set is for which site, changes the prefix of each table set. So In example does the following :
wp_options
wp_1_options
...
wp_x_options
In addition, the WordPress store the blog id in a special table called wp_blogs
So, from my shell script I run the following code :
mysql -uUSER -pPASS -e 'SELECT `blog_id` AS `ID`, `path` AS `Slug` FROM `wp`.`wp_blogs`'
and I am getting the following results
+----+---------------------------+
| ID | Slug |
+----+---------------------------+
| 1 | / |
| 2 | /site-2-slug/ |
| 4 | /site-4-slug/ |
| 5 | /site-5-slug/ |
| 6 | /site-6-slug/ |
| 7 | /site-7-slug/ |
| 8 | /site-8-slug/ |
| 9 | /site-9-slug/ |
| 10 | /site-10-slug/ |
+----+---------------------------+
So, now the actual question is, how can I parse the MySql result line by line, in order to get the ID and the Slug information ?
Side note 1 : The whole script has been generated and run's somehow manually. I need now this information in order to automate even farther the exporting script.
*Side note 2 : The MySql executed via the Vagrant ssh like the following line : *
sudo vagrant ssh --command "mysql -uroot -proot -e 'SELECT blog_id FROM wp.wp_blogs'"
You could save the result in a file using INTO like below:
SELECT blog_id, path FROM wp.wp_blogs
INTO OUTFILE '/tmp/blogs.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
And then you could process it line by line either usingsed/awk/simple while loop. Say you want to search for site and replace it with mysite, you could do something like:
awk -F',' '{print "Id: " $1 ", path: "$2}' /tmp/blogs.csv ##or simply cat the file.
I have a problem with writing in the variable from table in MySQL database.
DB="--user=$OSD_USERNAME \
--password=$OSD_PASSWORD \
--database=$OSD_DB -N";
mysql --user="$user" \
--password="$password" \
--database="$database" \
--execute="DROP DATABASE $user; CREATE DATABASE $database;"
id_customer = $(echo "SELECT id FROM customers WHERE customers.customer='John'"| mysql $DB)
My problem is that id_customer is still empty.
When I connect to mysql databases and put there this:
SELECT id FROM customers WHERE customers.customer='John'
then it give me table like this
+----+
| id |
+----+
| 1 |
+----+
Only empty, no errors? Try remove spaces between variable name and $
I have a text file with a list of IDs like this:
123
456
789
I would like to use them in queries to look up more information about these items. So far I have unsuccessfully tried:
awk '{print $1;}' 'pathto/file.txt' | xargs echo "SELECT * FROM table WHERE id='{}');" | mysql -u uname -p database
IDS=$(cat /path/to/file)
IDS=$(echo $IDS | sed 's/\s\s*/,/g')
#or IDS=$(echo $IDS | awk '{$1=$1}1' RS= OFS=,) on OSX
echo "SELECT * FROM table WHERE id in ($IDS)" | mysql -u uname -p database
Line 1 loads the ids into a variable, which allows to replace the line breaks easily in line 2, then replace the whitespace by commas to use an IN () construct.
Edits 1 and 2 see comments
Edit 3
on OSX (which I don't use or have access to) it seems sed doesn't like my regex. As the OP points out, using IDS=$(echo $IDS | awk '{$1=$1}1' RS= OFS=,) as line 2 works around this.
I am using this command:
mysql -u user -ppassword database -e "select distinct entityName,entitySource from AccessControl"
The output is like this:
+-----------------------+--------------+
| entityName | entitySource |
+-----------------------+--------------+
| low | Native |
| high | Native |
| All Groups | AD |
| Help Ser vices Group | AD |
| DEFAULT_USER_GROUP | Native |
| SYSTEM | Native |
| DEFAULT_BA_USER_GROUP | Native |
| soUsersGrp | Native |
+-----------------------+--------------+
My question is: how can I dynamically create an array of variables to store the values entityName and entitySource? What I need to use is use every value of entityName and entitySource to update another table.
Earlier I was trying to store the output in a file and access each line using awk, but that doesn't help because one line may contain multiple words.
Sure, this can be done. I'd like to second the idea that piping mysql to mysql in the shell is awkward, but I understand why it might need to be done (such as when piping mysql to psql or whatever).
mysql -qrsNB -u user -p password database \
-e "select distinct entityName,entitySource from AccessControl" | \
while read record; do
NAME="`echo $record|cut -d' ' -f 1`" # that's a tab delimiter
SOURCE="`echo $record|cut -d' ' -f 2`" # also a tab delimiter
# your command with $NAME and $SOURCE goes here ...
COMMAND="select trousers from namesforpants where entityName='${NAME}'" # ...
echo $COMMAND | mysql # flags ...
done
the -rs flags trim your output down so that you don't have to grok that table thing it gives you, -q asks that the result not be buffered, -B asks for batch mode, and -N asks to not have column names.
What you do with those variables is up to you; probably I would compose statements in that loop and feed those to your subsequent process rather than worry about interpolation and quotes as you have mentioned some of your data has spaces in it. Or you can write/append to a file and then feed that to your subsequent process.
As usual, the manual is your friend. I'll be your friend, too, but the manpage is where the answers are to this stuff. :-)
#!/usr/bin/env bash
mysql -u user -ppassword database -e "select distinct entityName,entitySource from AccessControl" | while read name source; do
echo "entityName: $name, entitySource: $source"
done
Please check it, I fixed it through exec.
[wcuser#localhost]$ temp=`exec mysql -h10.10.8.36 --port=3306 -uwcuser -pwcuser#123 paycentral -e "select endVersion from script_execution_detail where releaseNo='Release1.0' and versionPrefix='PS'"|tail -1`
[wcuser#localhost]$ echo $temp
19