I have a VM to train myself with chef solo.
Installed it and configured a kitchen. Configured my own VM as the only node in the kitchen.
Used Librarian to download the mysql cookbook and updated the runlist.
What is the command to use - to install the mysql on my node?
Thanks,
Liora
You can use the following command :
chef-solo -j JSON_ATTRIBS -c CONFIG
here
JSON_ATTRIB is the json data for the VM configuration
CONFIG is the configuration file which contains the runlist to be run on the VM
More help can be found using chef-solo --help
Related
I am newbie on using Ubuntu, I am trying to install apache-superset and successfully installed it by using Docker by directly sudo docker pull apache/superset, but I am stuck at adding Database Connector to the running superset
As for my local database I am using MySQL and I happen to use SSH Tunnel in localhost to access it in server. So I think at Docker container perspective this must be an "external" databases
What I have tried:
I installed mysqlclient from pip3
By following this references: https://devopsheaven.com/docker/devops/add-host/link/2017/10/04/connect-external-services-from-docker-container.html
I tried to type: sudo docker run -it mysql -h 192.168.100.1 -P 33063 -u czjovan --password=mypw cz_payment_merged but then i get:
2021-03-04 11:34:53+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.23-1debian10 started.
2021-03-04 11:34:53+00:00 [ERROR] [Entrypoint]: mysqld failed while attempting to check config
command was: mysqld -h 192.168.100.1 -P 33063 -u czjovan --password=mypw cz_payment_merged --verbose --help --log-bin-index=/tmp/tmp.EV6L0jrspQ
2021-03-04T11:34:53.402148Z 0 [ERROR] [MY-010124] [Server] Fatal error: Can't change to run as user 'czjovan' ; Please check that the user exists!
2021-03-04T11:34:53.403355Z 0 [ERROR] [MY-010119] [Server] Aborting
By Following this also: From inside of a Docker container, how do I connect to the localhost of the machine?
I tried to type: sudo docker run --rm it --network=host mysql mysql -h 127.0.0.1 but got error 2003 (hy000) can't connect to mysql server on 127.0.0.1
I tried to add mysqlconnector to sqlalchemy uri, but the driver not found
I am not an expert by setting these, I lack of Docker mechanism.. appreciate it if anyone willingly to direct me how to step by step resolving this..
UPDATE-------------------------------------------------:
-> Following Mustafa Guler to add -p 3306:3306, the mysql container now starts..
but I still cannot add database in Superset, what should I do next?
You need to provide the IP of the docker host machine. If you're using default networking, use the static IP 172.17.0.1 for the host.
Alternatively you can try host.docker.internal as the hostname.
You need to define user and pass as environment to create new user during start of mysql container. Maybe problem is user and pass MYSQL_USER, MYSQL_PASSWORD Also you do not expose 3306 port on docker run. Please use -p 3306:3306
I found out that installing Superset manually from scratch than using Superset from Docker Container are more a solution to me, since configuring Superset from docker manually can be a little cumbersome, what I did:
#-- Install Superset and MySQL Locally from Scratch ---#
sudo pip install apache-superset (
sudo apt install mysql-client-core-8.0
sudo apt install mysql-server
sudo pip install mysqlclient (for Superset Database Connectors)
#NOTE: there is also a package called 'superset' in pip, in my case i uninstalled this to ensure that only apache-superset is used
service mysql start
when mysql start, try to set password for the first time:
sudo mysqladmin -u root password
then to test it: mysql -u root -p, enter a created password
if it can enter mysql normally, the password set is successful,
CREATE DATABASE superset (this will save all config that superset progress will be saved)
Editing config.py in apache superset, which usually located on /usr/local/lib/python3.8/dist-packages/superset/config.py (this depends on pip installation from no 1),
8a) Edit the sqlalchemy uri part in config.py, so it can connect to a local installed mysql in ubuntu, to something like 'mysql://root:#localhost:3306/superset'
8b) Ensure that the database part in config.py is superset, or the same name with database name created in MySQL
After all database, sqlalchemy uri, and database name is prepared, its good to go to follow with Superset configs stated from: https://superset.apache.org/docs/installation/installing-superset-from-scratch
A) sudo superset db upgrade
B) sudo superset fab create-admin
C) sudo superset load examples
D) sudo superset init
E) sudo superset run -p 8088 --with-threads --reload --debugger (run this to start superset daily)
Run a Private SSH Tunnel Connecting to Database (Optional, in my case I use SSH Tunnel to access database)
And then I am able to add connection locally in Superset defining sqlalchemy uri
A) towards the SSH Tunnel mysql://(server_user):(server_password)#127.0.0.1:33063/(database_name)
(this is outside from local MySQL)
B) towards Local Installed MySQL 'mysql://root:rootpwd#127.0.0.1:3306/database_name'
I will recommend to use docker than manual installation by pip. Package dependencies especially flask are real problems.
Do not follow docker compose as beginner.
I had a problems using docker compose. Port is closed was annoying problem due to networking. Host.docker.internal doesn’t worked for me on Ubuntu 22 on Google cloud. I switched to manual installation and it was package versions hell on python 3.8. I would like to recommend to not follow official doc and use better approach with single docker image to start. Instead of running 5 containers by compose, run everything in one. Use official docker image, here image. Than modify docker file as follows to install custom db driver:
FROM apache/superset
USER root
RUN pip install mysqlclient
RUN pip install sqlalchemy-redshift
USER superset
Second step is to build new image based on docker file description. To avoid networking problems start both containers on same network (superset, your db) easier is to use host network. I used this on Google cloud, example as follow:
docker run -d --network host --name superset supers
The same command to start container with your database. —network host. This solved my problems. More about in whole step to step tutorial: medium or here blog
I have created a Linux VM instance in Google Compute Engine. I installed the JDK and Tomcat with the following commands using SSH.
sudo apt-get install default-jdk
sudo apt-get install tomcat8
I have a war file in my local machine. How can I move the war file from local machine to Compute Engine VM and run the war on the Tomcat server ?
I would recommend using Storage Buckets to store files. This way you can copy the same files to different VMs.
To use google storage from command line, first you would need to install gsutil.
To copy from your local machine to a bucket:
gsutil cp *.txt gs://my-bucket
To copy from a bucket to a VM, connect to your VM and run:
gsutil cp gs://my-bucket/*.txt .
More info about cp with gsutil on this link.
I have installed hue on my local ubuntu system and installed hadoop muti cluster system on two system.
Hadoop Version : 2.7.3
Hue Version : 3.12.0
Ozzie Version : 4.3.0
I am facing issue when I am running sqoop job process from mysql to import data from HDFS. I am getting following error.
Caused by: java.net.NoRouteToHostException: No Route to Host from Developer4/127.0.0.1 to cm:10020 failed on socket timeout exception: java.net.NoRouteToHostException: No route to host; For more details see: http://wiki.apache.org/hadoop/NoRouteToHost
HDFS url hdfs://master:9000
My /etc/hosts file like
192.168.1.149 master
127.0.0.1 developer4
192.168.1.161 slave
Please suggest me where I am doing wrong. Even ozzie command for start and stop command work properly on command line.
You have set Hadoop in your localhost system then you need to remove or modified below things in core-site.xml file.
mapreduce.jobhistory.address 0.0.0.0:10020 Host and port for Job History Server (default 0.0.0.0:10020)
After that you need to run jobhistoryservice with below command.
sbin/mr-jobhistory-daemon.sh --config /home/developer4/hadoop-2.7.3/etc start historyserver
After this command port is enable on your localhost and hope this will help you.
I started working on ejabberd recently and ran into a problem while connecting with mysql.
I had installed ejabberd using the command line on Ubuntu - sudo apt-get install ejabberd
I had referenced the following tutorial for setting up ejabberd - Install ejabberd on Ubuntu
Now, in order to connect to Mysql I have done the following changes in ejabberd.yml -
auth_method: odbc
odbc_type: mysql
odbc_server: "localhost"
odbc_database: "ejabberd"
odbc_username: "ejabberd"
odbc_password: "password"
##
## If you want to specify the port:
odbc_port: 3306
But on restarting the service, I get this error
[critical] <0.38.0>#ejabberd:exit_or_halt:133 failed to start application 'p1_mysql': {error,
{"no such file or directory",
"p1_mysql.app"}}
I saw a few links where it was mentioned to use ./configure --enable-odbc --enable-mysql to solve this issue(works when you install ejabberd by compiling the source), but since I have not compiled the source code to install ejabberd (like I mentioned above), I am unable to do this. How do I solve this issue. Thanks in advance.
To use ejabberd with Mysql in Ubuntu you need to install the erlang-p1-mysql package:
sudo apt install erlang-p1-mysql
I had the same issue and that solved it.
Compile it from source and put resulting p1_mysql* files in the installed ejabberd's ebin directory, e.g.:
$ ./configure --enable-mysql
$ make
$ cp deps/p1_mysql/ebin/* /path/to/running/ejabberd/ebin/
i am trying to setup mysql-proxy on ubuntu on amazon ec2
i have done following:
sudo apt-get install mysql-proxy --yes
vi /etc/default/mysql-proxy
i put following content on "/etc/default/mysql-proxy"
ENABLED="true"
OPTIONS="--proxy-lua-script=/usr/share/mysql-proxy/rw-splitting.lua
--proxy-address=127.0.0.1:3306
--proxy-backend-addresses=private_ip_of_another_ec2_db_server:3306,private_ip_of_another_ec2_db_server:3306"
also tied with "--proxy-address=private_ip_or_public_ip_of_proxy-server:3306 or 4040"
and "--proxy-backend-addresses=public_ip_of_another_ec2_db_server:3306,public_ip_of_another_ec2_db_server:3306"
after that i tried to connect proxy server from another pc using mysql like:
mysql -u some_user -pxxxxx -h proxy_server_ip
or
mysql -u some_user -pxxxxx -h proxy_server_ip -P 4040
but its not working
its showing error:
ERROR 2003 (HY000): Can't connect to MySQL server on 'ip' (10061)
i want to tell you can connect the db server remotely where i allowed remote connection to any host
i also tried /etc/init.d/mysql-proxy start or /etc/init.d/mysql-proxy restart but no result
just to inform you that /etc/init.d/mysql-proxy stop is showing failed
can anyone please help me to setup and configure mysql-proxy on ubuntu
===
Edit
i found some help from other question of stackoverflow and also according to a suggestion in the comments, have done following procedure. and it seems its working now.
i installed mysql-client and mysql-server locally(on proxy server)
then i tried to run mysql-proxy using following command:
mysql-proxy --proxy-backend-addresses=10.73.151.244:3306 --proxy-backend-addresses=10.73.198.7:3306 --proxy-address=:4040 --admin-username=root --admin-password=root --admin-lua-script=>/usr/lib/mysql-proxy/lua/admin.lua
then i tried to connect remotely to the proxy server and its working.
but it seems i need to run this command under screen because when i close the terminal proxy stops working.
Can you please tell me that do i need to run this command under screen or is there any other way to make it alive all time?
There is no need to install Mysql client or Mysql Server on your mysql-proxy.
Installing mysql-proxy does have "full daemon capabilities" compiled into it.
If your are running Ubuntu Server, you may wish to use an UPSTART service script.
This script can be copied into /etc/init/mysql-proxy.conf
# mysql-proxy.conf (Ubuntu 14.04.1) Upstart proxy configuration file for AWS RDS
# mysql-proxy - mysql-proxy job file
description "mysql-proxy upstart script"
author "shadowbq <shadowbq#gmail.com>"
# Stanzas
#
# Stanzas control when and how a process is started and stopped
# See a list of stanzas here: http://upstart.ubuntu.com/wiki/Stanzas#respawn
# When to start the service
start on runlevel [2345]
# When to stop the service
stop on runlevel [016]
# Automatically restart process if crashed
respawn
# Essentially lets upstart know the process will detach itself to the background
expect daemon
# Run before process
pre-start script
[ -d /var/run/mysql-proxy ] || mkdir -p /var/run/mysql-proxy
echo "starting mysql-proxy"
end script
# Start the process
exec /usr/bin/mysql-proxy --plugins=proxy --proxy-lua-script=/usr/share/mysql-proxy/rw-splitting.lua --log-level=debug --proxy-backend-addresses=private_ip_of_another_ec2_db_server:3306,private_ip_of_another_ec2_db_server:3306 --daemon --log-use-syslog --pid-file=/var/run/mysql-proxy/mysql-proxy.pid
In the above example I hard coded the AWS RDS server into script, instead of fiddling with defaults and config file
Install Upgraded version 0.8.5
Note:
apt repo does not have 0.8.5 so we need to download tar from mysql official site
Prerequisite :-
Create file /etc/default/mysql-proxy with following content
ENABLED="true"
OPTIONS="--defaults-file=/etc/mysql/mysql-proxy.cnf"
Installation Procedure :-
Download mysql-proxy 0.8.x
Untar in /usr/local
Update PATH environment with /usr/local/mysql-proxy-0.8.5-linux-debian6.0-x86-64bit/bin
vim /etc/environment (to update environment path)
cd /usr/local/mysql-proxy-0.8.5-linux-debian6.0-x86-64bit/bin
Run command sudo ./mysql-proxy --defaults-file=/etc/mysql/mysql-proxy.cnf
Sample mysql-proxy.cnf file
[mysql-proxy]
log-level=debug
log-file=/var/log/mysql-proxy.log
pid-file = /var/run/mysql-proxy.pid
daemon = true
--no-proxy = false
admin-username=ADMIN
admin-password=ADMIN
proxy-backend-addresses=RDS-ENDPOINT:RDS-PORT
admin-lua-script=/usr/lib/mysql-proxy/lua/admin.lua
proxy-address=0.0.0.0:4040
admin-address=localhost:4041
change host ip and port of RDS or mysql
connect to Mysql server via proxy with
mysql -h{proxy-host-ip} -P 4040 -u{mysql_username} -p