when i do rhc cartridge add mysql-5.5 --app NameApp the engine default of DB is MyISAM. But if i need engine InnoDB, how can i create db on OpenShift?
The default storage engine for mysql is set with an environment variable. Add the cartridge, then run the below to use InnoDB:
$ rhc env add OPENSHIFT_MYSQL_DEFAULT_STORAGE_ENGINE=InnoDB -a NameApp
$ rhc app-restart NameApp
See the mysql cartridge configuration where this is set:
https://github.com/openshift/origin-server/blob/master/cartridges/openshift-origin-cartridge-mysql/conf/my.cnf.erb#L39
Related
I need the federated engine in the entrypoint for pre populate database in the container. To enable the federated engine I wrote a bash script to manual insert the federated command to the my.cnf, but getting error below
2021-07-20 16:14:22+00:00 [Note] [Entrypoint]: /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/_enable_federated.sh
sed: couldn't open temporary file /etc/mysql/sedHjVLxp: Permission denied
_enable_federated.sh
#!/bin/bash
sed -i '/\[mysqld\]/a federated' /etc/mysql/my.cnf
May I know what is the correct way to enable federated engine
The easiest way to install the federated storage engine is to run the container with the command line argument --plugin-load-add=ha_federatedx
Alternately for a configuration file based approach use the (ref: "Using a custom MariaDB configuration file" in documenation):
$ cat config/federated.cnf
[mariadbd]
plugin-load-add=ha_federated
$ podman run -v=./config/:/etc/mysql/conf.d/ -e MARIADB_ALLOW_ROOT_EMPTY_PASSWORD=1 mariadb:10.5
The final way is to use in /docker-entrypoint-initdb.d/ a federated.sql script:
INSTALL SONAME 'ha_federatedx';
I am newbie on using Ubuntu, I am trying to install apache-superset and successfully installed it by using Docker by directly sudo docker pull apache/superset, but I am stuck at adding Database Connector to the running superset
As for my local database I am using MySQL and I happen to use SSH Tunnel in localhost to access it in server. So I think at Docker container perspective this must be an "external" databases
What I have tried:
I installed mysqlclient from pip3
By following this references: https://devopsheaven.com/docker/devops/add-host/link/2017/10/04/connect-external-services-from-docker-container.html
I tried to type: sudo docker run -it mysql -h 192.168.100.1 -P 33063 -u czjovan --password=mypw cz_payment_merged but then i get:
2021-03-04 11:34:53+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.23-1debian10 started.
2021-03-04 11:34:53+00:00 [ERROR] [Entrypoint]: mysqld failed while attempting to check config
command was: mysqld -h 192.168.100.1 -P 33063 -u czjovan --password=mypw cz_payment_merged --verbose --help --log-bin-index=/tmp/tmp.EV6L0jrspQ
2021-03-04T11:34:53.402148Z 0 [ERROR] [MY-010124] [Server] Fatal error: Can't change to run as user 'czjovan' ; Please check that the user exists!
2021-03-04T11:34:53.403355Z 0 [ERROR] [MY-010119] [Server] Aborting
By Following this also: From inside of a Docker container, how do I connect to the localhost of the machine?
I tried to type: sudo docker run --rm it --network=host mysql mysql -h 127.0.0.1 but got error 2003 (hy000) can't connect to mysql server on 127.0.0.1
I tried to add mysqlconnector to sqlalchemy uri, but the driver not found
I am not an expert by setting these, I lack of Docker mechanism.. appreciate it if anyone willingly to direct me how to step by step resolving this..
UPDATE-------------------------------------------------:
-> Following Mustafa Guler to add -p 3306:3306, the mysql container now starts..
but I still cannot add database in Superset, what should I do next?
You need to provide the IP of the docker host machine. If you're using default networking, use the static IP 172.17.0.1 for the host.
Alternatively you can try host.docker.internal as the hostname.
You need to define user and pass as environment to create new user during start of mysql container. Maybe problem is user and pass MYSQL_USER, MYSQL_PASSWORD Also you do not expose 3306 port on docker run. Please use -p 3306:3306
I found out that installing Superset manually from scratch than using Superset from Docker Container are more a solution to me, since configuring Superset from docker manually can be a little cumbersome, what I did:
#-- Install Superset and MySQL Locally from Scratch ---#
sudo pip install apache-superset (
sudo apt install mysql-client-core-8.0
sudo apt install mysql-server
sudo pip install mysqlclient (for Superset Database Connectors)
#NOTE: there is also a package called 'superset' in pip, in my case i uninstalled this to ensure that only apache-superset is used
service mysql start
when mysql start, try to set password for the first time:
sudo mysqladmin -u root password
then to test it: mysql -u root -p, enter a created password
if it can enter mysql normally, the password set is successful,
CREATE DATABASE superset (this will save all config that superset progress will be saved)
Editing config.py in apache superset, which usually located on /usr/local/lib/python3.8/dist-packages/superset/config.py (this depends on pip installation from no 1),
8a) Edit the sqlalchemy uri part in config.py, so it can connect to a local installed mysql in ubuntu, to something like 'mysql://root:#localhost:3306/superset'
8b) Ensure that the database part in config.py is superset, or the same name with database name created in MySQL
After all database, sqlalchemy uri, and database name is prepared, its good to go to follow with Superset configs stated from: https://superset.apache.org/docs/installation/installing-superset-from-scratch
A) sudo superset db upgrade
B) sudo superset fab create-admin
C) sudo superset load examples
D) sudo superset init
E) sudo superset run -p 8088 --with-threads --reload --debugger (run this to start superset daily)
Run a Private SSH Tunnel Connecting to Database (Optional, in my case I use SSH Tunnel to access database)
And then I am able to add connection locally in Superset defining sqlalchemy uri
A) towards the SSH Tunnel mysql://(server_user):(server_password)#127.0.0.1:33063/(database_name)
(this is outside from local MySQL)
B) towards Local Installed MySQL 'mysql://root:rootpwd#127.0.0.1:3306/database_name'
I will recommend to use docker than manual installation by pip. Package dependencies especially flask are real problems.
Do not follow docker compose as beginner.
I had a problems using docker compose. Port is closed was annoying problem due to networking. Host.docker.internal doesn’t worked for me on Ubuntu 22 on Google cloud. I switched to manual installation and it was package versions hell on python 3.8. I would like to recommend to not follow official doc and use better approach with single docker image to start. Instead of running 5 containers by compose, run everything in one. Use official docker image, here image. Than modify docker file as follows to install custom db driver:
FROM apache/superset
USER root
RUN pip install mysqlclient
RUN pip install sqlalchemy-redshift
USER superset
Second step is to build new image based on docker file description. To avoid networking problems start both containers on same network (superset, your db) easier is to use host network. I used this on Google cloud, example as follow:
docker run -d --network host --name superset supers
The same command to start container with your database. —network host. This solved my problems. More about in whole step to step tutorial: medium or here blog
I found that there is an official MySQL image for docker here. Unfortunately, I am unable to start the container with a host volume. I have tried the following script.
DB_ROOT_PASS="myPassword"
DB_NAME="myDatabase"
DATA_DIR="$HOME/mysql_data"
mkdir -p $DATA_DIR
mkdir -p $DATA_DIR/conf.d
cd $DATA_DIR
wget -O my.cnf http://pastebin.com/raw.php?i=aV4pXRQD
sudo docker run \
-e MYSQL_ROOT_PASSWORD=$DB_ROOT_PASS \
-e MYSQL_DATABASE=$DB_NAME \
-p 3306:3306 \
-v $DATA_DIR:/etc/mysql \
mysql
which results in the following output:
Running mysql_install_db ...
Installing MySQL system tables...2015-02-14 07:49:52 0 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
2015-02-14 07:49:52 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
OK
Filling help tables...2015-02-14 07:49:55 0 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
2015-02-14 07:49:55 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
OK
To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system
PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:
/usr/bin/mysqladmin -u root password 'new-password'
/usr/bin/mysqladmin -u root -h 2efc65e86051 password 'new-password'
Alternatively you can run:
/usr/bin/mysql_secure_installation
which will also give you the option of removing the test
databases and anonymous user created by default. This is
strongly recommended for production servers.
See the manual for more instructions.
You can start the MySQL daemon with:
cd /usr ; /usr/bin/mysqld_safe &
You can test the MySQL daemon with mysql-test-run.pl
cd mysql-test ; perl mysql-test-run.pl
Please report any problems at http://bugs.mysql.com/
The latest information about MySQL is available on the web at
http://www.mysql.com
Support MySQL by buying support/licenses at http://shop.mysql.com
WARNING: Found existing config file /usr/my.cnf on the system.
Because this file might be in use, it was not replaced,
but was used in bootstrap (unless you used --defaults-file)
and when you later start the server.
The new default config file was created as /usr/my-new.cnf,
please compare it with your file and take the changes you need.
WARNING: Default config file /etc/mysql/my.cnf exists on the system
This file will be read by default by the MySQL server
If you do not want to use this, either remove it, or use the
--defaults-file argument to mysqld_safe when starting the server
Finished mysql_install_db
2015-02-14 07:49:57 0 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
2015-02-14 07:49:57 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
I have also tried without creating the conf.d directory or the my.cnf files which also result in other errors.
Why is a volume necessary?
A host volume provides "near metal" disk performance as well as data persistance which is important with a MySQL database. Thus backing up the data from the container, or using the "data container" pattern is not desired.
Question:
How can I update the script in order to start the Docker MySQL container with a host volume such that the MySQL service starts correctly? Perhaps passing an init script or adding more data to the volume?
Context
Ubuntu 14.04 64bit server
Docker version 1.5.0, build a8a31ef
There were several problems, the primary one being that the original config file that was being downloaded had a bind-address setting that resulted in the mysql service not being able to be connected to from outside the host. I have since updated the setting (commented it out) and gotten the service running correctly, yet service mysql status still states MySQL Community Server 5.6.23 is not running. even though it is. I am 99% sure this is because of the "containerization" effect.
Also, the data is not kept at /etc/mysql, so to achieve persistence and higher performance, we need to add another volume. The finalized working script is as such:
DB_ROOT_PASS="myPassword"
DB_NAME="myDatabase"
CONFIG_DIR="$HOME/mysql_config"
DATA_DIR="$HOME/mysql_data"
mkdir -p $DATA_DIR
mkdir -p $CONFIG_DIR
mkdir -p $CONFIG_DIR/conf.d
cd $CONFIG_DIR
sudo wget -O my.cnf http://pastebin.com/raw.php?i=aV4pXRQD
sudo docker run \
-e MYSQL_ROOT_PASSWORD=$DB_ROOT_PASS \
-e MYSQL_DATABASE=$DB_NAME \
-p 3306:3306 \
-v $CONFIG_DIR:/etc/mysql \
-v $DATA_DIR:/var/lib/mysql \
-d \
mysql
The reason you are seeing the message
MySQL Community Server 5.6.23 is not running.
is because the command ps is not loaded in this container. The service mysql status command (and all of the service status commands) depend on ps to check status.
When I run service mysql status the full message I get is:
/etc/init.d/mysql: line 41: ps: command not found
MySQL Community Server 5.6.23 is not running.
And indeed if I try running ps directly I get
bash: ps: command not found
Not certain why they did not provide ps in this container, might be worth opening an issue on it.
I am trying to install MySQL 5.6 in amazon linux machine. by using following link.
after completing following steps.
sudo yum localinstall http://repo.mysql.com/mysql-community-release-el6-3.noarch.rpm
sudo yum install mysql-community-server
when i am starting mysql services by this command.
sudo service mysqld start
I am getting MySQL Daemon failed to start error.
Remove old mySql
sudo yum remove mysql mysql-server mysql-common mysql-client
remove all pre-installed packages. You can get a list:
rpm -qa | grep -i mysql
Then uninstall all of them. For example:
rpm -e mysql libmysqlclient15-5.0.94-0.2.4.1 <and so on>
Completely remove the /var/lib/mysql folder
cd /var/lib
rm -rf mysql
Installing fresh mySql 5.6 =>
wget http://dev.mysql.com/get/Downloads/MySQL-5.6/MySQL-5.6.23-1.el7.x86_64.rpm-bundle.tar
tar -xvf MySQL-5.6.23-1.el7.x86_64.rpm-bundle.tar
sudo yum -y install MySQL-client-5.6.23-1.el7.x86_64.rpm
sudo yum install MySQL-shared-compat-5.6.23-1.el7.x86_64.rpm
sudo yum install MySQL-server-5.6.23-1.el7.x86_64.rpm
Starting from 2015 Sep, Mysql 5.6 server is now natively available through Amazon yum repos. More info # https://aws.amazon.com/blogs/aws/now-available-amazon-linux-ami-2015-09/
You can now simply install Mysql 5.6 server using
sudo yum install mysql56-server
Then you can simply start/stop/status look using the regular service commands
service mysqld start ( restart | stop | status | )
I couldn't figure out the default root password in Amazon AMI then I simply reset the root user password by starting the mysqld service with skip grant tables
mysqld_safe --skip-grant-tables start
mysql -u root
update user set password=PASSWORD('[New Password]') where User='root';
OR
/usr/libexec/mysql56/mysqladmin -u root password 'new password'
Also, you need to run the following command to upgrade the MySQL database to 5.6
sudo mysql_upgrade -u root -p
I met the same problem on my micro instance. Check your mysql logs
If the issue is because of
InnoDB: Cannot allocate memory for the buffer pool
Adding a swap page might solve the problem. It solved for me.
You can follow this setup
http://www.prowebdev.us/2012/05/amazon-ec2-linux-micro-swap-space.html
I copied the contents if the page does not loads
Amazon EC2 Micro Instance Swap Space - Linux
I have a Amazon EC2 Linux Micro instance. Since Micro instances have only 613MB of memory, MySQL crashed every now and then. After a long search about MySQL, Micro Instance and Memory Managment I found out there is no default SWAP space for Micro instance. So if you want to avoid the crash you may need to setup a swap space for your micro instance. Actually performance wise is better to enable swap.
Steps below show how to make a swap space for your Micro instance. I assume you have AWS Account with a Micro instance running.
Run dd if=/dev/zero of=/swapfile bs=1M count=1024
Run mkswap /swapfile
Run swapon /swapfile
Add this line /swapfile swap swap defaults 0 0 to /etc/fstab
Step 4 is needed if you would like to automatically enable swap file after each reboot.
Some useful command related to SWAP space:
$ swapon -s
$ free -k
$ swapoff -a
$ swapon -a
When I execute a query in MySQL it returns an error saying that InnoDB is not enabled. When I clicked the storage engine, the InnoDB was disabled.
How do I enable InnoDB?
I faced a similar situation where InnoDB got disabled after a mysql-server upgrade. The query "show engines" didn't display Innodb. Following this link fixed the issue for me.
/etc/init.d/mysql stop
cd /var/lib/mysql/
ls ib_logfile*
mv ib_logfile0 ib_logfile0.bak
mv ib_logfile1 ib_logfile1.bak
/etc/init.d/mysql restart
You need to enable it in my.cnf file, then restart your server:
http://dev.mysql.com/doc/refman/5.1/en/innodb-parameters.html#option_mysqld_innodb
Or you can load an InnoDB plugin during runtime:
https://docs.oracle.com/cd/E19078-01/mysql/mysql-refman-5.1/storage-engines.html#replacing-builtin-innodb
In my.ini (located in MySQL folder) put a # sign before 'skip-innodb' to disable this command. Then restart mysql. This will enable InnoDB engine.
If your InnoDB gets disabled after a mysql-server upgrade what you have to do Initially is to set plugin-load of at server startup using
[mysqld]
plugin-load="myplugin_1=myplugin_1.so;myplugin_2=myplugin_2.so";
And then specify the pathname to the plugin_dir(plugin directory) it can be done by following changes in the my.cnf file
[mysqld]
ignore-builtin-innodb
plugin-load=innodb=ha_innodb_plugin.so
plugin_dir=/path/to/plugin/directory