I read the documentation about running metabase with docker from here and setting metabase up with mysql from here and followed all the instruction as per the documentation.
Here is the docker-compose.yml file I used to run metabase:
# docker-compose.yml
version: '3'
services:
metabase:
image: metabase/metabase:v0.39.1
environment:
- MB_DB_TYPE=mysql
- MB_DB_DBNAME=metabase
- MB_DB_PORT=3306
- MB_DB_USER=secretuser
- MB_DB_PASS=secretword
- MB_DB_HOST=secrethost
ports:
- 3000:3000
When I run docker-compose up I get the following error.
metabase_1 | 2021-05-17 08:27:59,084 ERROR metabase.core :: Metabase Initialization FAILED
metabase_1 | liquibase.exception.MigrationFailedException: Migration failed for change set migrations/000_migrations.yaml::1::agilliland:
metabase_1 | Reason: liquibase.exception.DatabaseException: (conn=32027) Table 'core_organization' already exists [Failed SQL: CREATE TABLE `metabase`.`core_organization` (`id` INT AUTO_INCREMENT NOT NULL, `slug` VARCHAR(254) NOT NULL, `name` VARCHAR(254) NOT NULL, `description` TEXT NULL, `logo_url` VARCHAR(254) NULL, `inherits` BIT(1) NOT NULL, CONSTRAINT `PK_CORE_ORGANIZATION` PRIMARY KEY (`id`), UNIQUE (`slug`)) ENGINE InnoDB CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci]
metabase_1 | at liquibase.changelog.ChangeSet.execute(ChangeSet.java:637)
..
..
metabase_1 | Caused by: java.sql.SQLException: Table 'core_organization' already exists
metabase_1 | at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.readErrorPacket(AbstractQueryProtocol.java:1688)
metabase_1 | at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.readPacket(AbstractQueryProtocol.java:1550)
metabase_1 | at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:1513)
metabase_1 | at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:256)
metabase_1 | ... 57 more
metabase_1 | 2021-05-17 08:27:59,109 INFO metabase.core :: Metabase Shutting Down ...
I also tried to run directly without using compose file using this command:
docker run -d -p 3000:3000 -e "MB_DB_TYPE=mysql" -e "MB_DB_DBNAME=metabase" \
-e "MB_DB_PORT=3306" -e "MB_DB_USER=secretuser" -e 'MB_DB_PASS=secretpass' \
-e "MB_DB_HOST=secrethost" --name metabase metabase/metabase:v0.39.1
I get the exactly same error as with the compose file.
On further searches I found a similar issue in github stating that "Table 'core_organization' already exists", however, they claim that it is already fixed but I'm still having the issue. In the issue page they suggest: java -jar metabase.jar migrate force. How can I run this from docker ?
NOTE: I have tried to drop all the table and start with fresh schema multiple times and it didn't work as well.
NOTE: the database I use is AWS RDS, just to make sure that it's not related with AWS.
The root of the problem was that I didn't have permissions to ALTER and REFERENCE in AWS RDS. Once I got those permissions everything worked fine.
Related
I'm installing MySQL 8 on an Arm EC2 instance and I cannot get it to work with remote connections. Here's the steps I do to install MySQL:
yum -y install https://dev.mysql.com/get/mysql80-community-release-el7-3.noarch.rpm --nogpgcheck
yum -y install mysql-community-server --nogpgcheck
I then start MySQL with:
systemctl start mysqld
Then I login as root to MySQL and create a sample database with:
CREATE DATABASE gtfs DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
USE gtfs;
CREATE TABLE agency (
agency_id VARCHAR(255) NOT NULL PRIMARY KEY,
agency_name VARCHAR(255),
agency_url VARCHAR(255),
agency_timezone VARCHAR(255),
agency_lang VARCHAR(255)
);
INSERT INTO agency VALUES ('id', 'name', 'url', 'timezone', 'lang');
So now I have a database created. If I try to connect from a client as root I get ER_HOST_NOT_PRIVILEGED. I've followed along here and created a new user with:
CREATE USER 'lambda'#'%' IDENTIFIED BY 'Password1!';
GRANT ALL ON *.* TO 'lambda'#'%';
and tried it out. I first get ER_NOT_SUPPORTED_AUTH_MODE which I can fix with:
UPDATE mysql.user SET plugin='mysql_native_password' WHERE User='lambda';
And then I'm back getting ER_HOST_NOT_PRIVILEGED with my lambda user. I've also added
bind-address=0.0.0.0
to my /etc/my.cnf file and restarted MySQL with systemctl restart mysqld and it still isn't working.
Running mysql -V outputs
mysql Ver 8.0.28 for Linux on aarch64 (MySQL Community Server - GPL)
Any help is massively appreciated. I'm tearing my hair out at this point. I've checked and I have port 3306 open on the EC2 instance. I cannot figure this out.
I have such a problem, I'm trying to do master-slave replication according to this tutorial https://developpaper.com/master-slave-replication-of-mysql-based-on-docker/ .
There is no problem with the building of the image, but there is a problem with running this container. I can't get access to the mysql commands of mysql container in docker, it seems to be built from mysql (image from docker hub), but can not be started.
There are the following files:
Dockerfile:
#Using MySQL image to create a new image
FROM mysql:latest
ENV MYSQL_ROOT_PASSWORD Kohc9hai
COPY start.sh /mysql/start.sh
COPY my.cnf /etc/mysql/my.cnf
COPY init.sql /mysql/init.sql
EXPOSE 6603
CMD ["sh", "/mysql/start.sh"]
init.sql:
--Create data_ Copy database
DROP DATABASE IF EXISTS `data_copy`;
CREATE DATABASE `data_copy` /*!40100 DEFAULT CHARACTER SET utf8mb4 collate utf8mb4_general_ci */;
--Create person table
USE `data_copy`;
DROP TABLE IF EXISTS `person`;
CREATE TABLE `person` (
`id` int(32) NOT NULL,
`name` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
my.cnf:
[mysqld]
log-bin = mysql-bin
server-id = 1
start.sh:
#!/bin/sh
echo "start MySQL"
service mysql start
sleep 5
echo "initialize database"
mysql -uroot -pKohc9hai < /mysql/init.sql
echo "initialization complete"
tail -f /dev/null
By getting deeper, I think that I determined that the problem is in the file start.sh (Maybe I'm wrong).
There are logs of this container:
start MySQL
mysql: unrecognized service
initialize database
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
initialization complete
Maybe someone can help me with the solution to this problem.
I'v just started to study Docker.
mysql: unrecognized service
From next, you could see there is no mysql service in container, so you surely failure:
$ docker run --rm mysql ls /etc/init.d
hwclock.sh
In fact, mysql image use next to start mysql service:
exec gosu mysql "$BASH_SOURCE" "$#"
So you need to follow the same way.
BUT, for your scenario, looks you just customize start.sh to init some sql, while it's in fact already supported in official image:
Initializing a fresh instance
When a container is started for the first time, a new database with the specified name will be created and initialized with the provided configuration variables. Furthermore, it will execute files with extensions .sh, .sql and .sql.gz that are found in /docker-entrypoint-initdb.d. Files will be executed in alphabetical order. You can easily populate your mysql services by mounting a SQL dump into that directory and provide custom images with contributed data. SQL files will be imported by default to the database specified by the MYSQL_DATABASE variable.
So, what you need to do is next in Dockerfile (Maybe need to specify MYSQL_DATABASE also):
COPY init.sql /docker-entrypoint-initdb.d
The document is quite old (docker version 1.13.1! Now 19.03.13!!). No special setting is required to use MySQL image. All you need to do is:
docker run --name foo -d -p 6603:6603 mysql:latest
Please help me to solve this problem:
I am using docker to create a mysql container and initialize a database in it automatically based on mysql:8.0.19. But when I get into the container and enter the pass word after 'mysql -u root -p', it is said that:
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock'
There are four files: Dockerfile, schema.sql, privileges.sql, setup.sh
Dockerfile:
FROM mysql:8.0.19
# allow no password
ENV MYSQL_ALLOW_EMPTY_PASSWORD yes
# copy file into container
COPY setup.sh /mysql/setup.sh
COPY schema.sql /mysql/schema.sql
COPY privileges.sql /mysql/privileges.sql
# exec these command when container start up
CMD ["sh", "/mysql/setup.sh"]
setup.sh:
#!/bin/bash
echo 'checking mysql status.'
service mysql status
echo '1.start mysql....'
service mysql start
sleep 3
service mysql status
echo '2.start importing data....'
mysql < /mysql/schema.sql
echo '3.end importing data....'
sleep 3
service mysql status
echo '4.start changing password....'
mysql < /mysql/privileges.sql
echo '5.end changing password....'
sleep 3
service mysql status
echo 'mysql is ready'
tail -f /dev/null
privileges.sql:
use mysql;
select host, user, plugin from user;
ALTER USER 'root'#'localhost' IDENTIFIED BY '123456';
FLUSH PRIVILEGES;
schema.sql:
create database `collector`;
use collector;
DROP TABLE IF EXISTS `EVENT`;
CREATE TABLE `EVENT` (
`ID` VARCHAR(50) NOT NULL,
`EventType`VARCHAR(50) DEFAULT NULL,
`EventID` VARCHAR(50) DEFAULT NULL,
`EpcID` VARCHAR(50) DEFAULT NULL,
`TimeStamp` FLOAT DEFAULT NULL,
PRIMARY KEY (`ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
INSERT INTO `EVENT` (`ID`, `EventType`, `EventID`, `EpcID`, `TimeStamp`)
VALUES('0','0','0','0',0.0);
I think privileges.sql might be the one that causing the problem, but i have no idea how to solve it.
By the way I use:
docker build . -t mysql_service:1.0
docker run -p 3306:3306 --name mysql_service -e MYSQL_ROOT_PASSWORD=123456 -d mysql_service:1.0
to build and run.
To set a root password use MYSQL_ROOT_PASSWORD=123456.
MYSQL_DATABASE=collector is also available to create the database and use it as the default.
FLUSH PRIVILEGES aren't needed for ALTER USER.
The docker container doesn't have a init so service ... commands won't work. Also the mysqld isn't running in the build stage.
https://hub.docker.com/_/mysql/ under Initializing a fresh instance you can put SQL files in /docker-entrypoint-initdb.d to initialize the database.
CREATE TABLE `revenue_daily` ( `wallet` varbinary(100) NOT NULL DEFAULT '0',
`tc_access` varbinary(100) NOT NULL DEFAULT '0',
`tc_short` varbinary(100) NOT NULL DEFAULT '0',
`total_toll_collection` varbinary(100) GENERATED ALWAYS AS (`wallet` + `tc_access`) VIRTUAL NOT NULL,
`cash_collection` varbinary(100) GENERATED ALWAYS AS (`total_toll_collection` - `tc_short`) VIRTUAL NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=ascii;
That table has generated column.I backed up database structure with data and when i am restoring same .sql file then error occur.
Error is:-
ERROR 3105 (HY000) at line 262: The value specified for generated column 'total_toll_collection' in table 'revenue_daily' is not allowed.
I am using mysql version:-
sunilp#sunilp ~> mysql --version
mysql: [Warning] World-writable config file '/etc/mysql/mysql.conf.d mysqld.cnf' is ignored.
mysql Ver 14.14 Distrib 5.7.18, for Linux (x86_64) using EditLine wrapper
This is a problem when using mysqldump from MariaDB with virtual generated columns.
MariaDB's mysqldump apparently dumps the generated values, but MySQL only accepts DEFAULT as value for a virtual generated column.
It seems like you need to use MySQL's mysqldump to correctly dump and restore virtual generated columns on a MySQL server.
The bug was also reported here.
What I do as a workaround, is replace the virtual column in the dump:
sed -i 's/GENERATED ALWAYS AS .* VIRTUAL/NOT NULL/' mydump.sql
then restore the dump, then drop/add the generated column again:
mysql -e "ALTER TABLE foo DROP COLUMN bar;\
ALTER TABLE foo ADD COLUMN bar VARCHAR(255) AS ...;"
I also posted this answer here.
I had the same problem dumping it from mariadb using adminer (https://www.adminer.org/)
In my case I solved it doing dump with phpmyadmin. It worked for me.
I don't know reason but adminer dump generated values columns. It shouldn't because that value should be generated when rows are inserted.
I didn't try what happens when using mysqldump from command line...
This question is based on this thread.
I run unsuccessfully
sudo mysql
\. /users/cs/SO_db/posts.sql
I get the error
ERROR 1146 (42S02): Table 'personal.posts' doesn't exist
MySQL's manual says
A five-character SQLSTATE value
('42S02'). The values are specified by
ANSI SQL and ODBC and are more
standardized. Not all MySQL error
numbers are mapped to SQLSTATE error
codes. The value 'HY000' (general
error) is used for unmapped errors.
and
Error: 1146 SQLSTATE: 42S02
(ER_NO_SUCH_TABLE)
Message: Table '%s.%s' doesn't exist
How can you solve the error message?
The SQL script you have loaded makes reference to a database and/or table which does not exist in the database.
Typically one would not call the mysql tool with sudo, as the system user privileges are different from MySQL users.
To execute an SQL script through mysql I would try something like:
cat somefile.sql | mysql -u <mysqluser> -p <mysqldb>
This command would load 'somefile.sql' into mysql tool, connecting to a MySQL server on localhost as user <mysqluser> and selecting the database <mysqldb>. The mysql tool will prompt for <mysqluser>'s access password before executing the script.
As I mentioned in the post you referenced, you NEED to create the tables first.
Peek at the XML or the SQL output on what columns you need. e.g. here is a table that can hold the output from badges.xml (I don't have the others available right now..)
CREATE TABLE `badges` (
`Id` int(11) NOT NULL default '0',
`UserId` int(11) not NULL,
`Date` datetime not NULL,
`Name` varchar(32) not NULL,
PRIMARY KEY (`Id`),
KEY `Date` (`Date`),
KEY `UserId` (`UserId`)
) ;
Have you actually created the database 'personal' and the table 'posts'?
You might want to try something like:
mysql -h localhost -u <user> -p<password> -D personal < /users/cs/SO_db/posts.sql
Your posts.sql contains some statements referencing a posts table, which doesn't exist in your personal schema.
To "solve" the error, ensure the table is created!
probably your sql file doesn't include the "create table" command.