MySQL "create table" does not work in docker image - mysql

I make docker image with mysql.
I created init.sql file to create table when starting docker file.
but after building docker image, table is not created.
I want to create table via DockerFile or docker-compose.
Here is source code in github.
https://github.com/jpskgc/article
I created int.sql file and docker-compose.yml to use this init.sqlfile.
article
 ├ db
 │ └ init.sql
 │
 └ docker-compose.yml
//docker-compose.yml
version: '3'
services:
db:
image: mysql
ports:
- '3306:3306'
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: article
MYSQL_USER: docker
MYSQL_PASSWORD: docker
volumes:
- ./db:/docker-entrypoint-initdb.d
//init.sql
CREATE DATABASE IF NOT EXISTS article;
use article;
CREATE TABLE IF NOT EXISTS `articles` (`id` int NOT NULL AUTO_INCREMENT PRIMARY KEY, uuid varchar(36), `title` VARCHAR(100) NOT NULL,`content` TEXT NOT NULL) ENGINE=InnoDB DEFAULT CHARSET=utf8;
create table IF NOT EXISTS images (id int AUTO_INCREMENT NOT NULL PRIMARY KEY, article_uuid varchar(36), image_name varchar(50));
I want to create table in the process of docker-compose up --build.
After command docker-compose up --build and assess to 'http://localhost:3000/', there is error in backend, which suggesting that table does not exist.
api_1 | 2019/07/31 06:29:58 [Recovery] 2019/07/31 - 06:29:58 panic recovered:
api_1 | GET /api/articles HTTP/1.1
api_1 | Host: localhost:2345
api_1 | Accept: application/json, text/plain, */*
api_1 | Accept-Encoding: gzip, deflate, br
api_1 | Accept-Language: en-US,en;q=0.9
api_1 | Connection: keep-alive
api_1 | Origin: http://localhost:3000
api_1 | Referer: http://localhost:3000/
api_1 | User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36
api_1 |
api_1 |
api_1 | Error 1146: Table 'article.articles' doesn't exist
api_1 | /app/main.go:90 (0xb1c6d1)
api_1 | main.func2: panic(err.Error())
api_1 | /go/src/github.com/gin-gonic/gin/context.go:147 (0x8f97c9)
api_1 | (*Context).Next: c.handlers[c.index](c)
api_1 | /go/src/github.com/gin-gonic/gin/recovery.go:83 (0x90d259)
api_1 | RecoveryWithWriter.func1: c.Next()
api_1 | /go/src/github.com/gin-gonic/gin/context.go:147 (0x8f97c9)
api_1 | (*Context).Next: c.handlers[c.index](c)
api_1 | /go/src/github.com/gin-gonic/gin/logger.go:240 (0x90c300)
api_1 | LoggerWithConfig.func1: c.Next()
api_1 | /go/src/github.com/gin-gonic/gin/context.go:147 (0x8f97c9)
api_1 | (*Context).Next: c.handlers[c.index](c)
api_1 | /go/src/github.com/gin-gonic/gin/gin.go:391 (0x9036c9)
api_1 | (*Engine).handleHTTPRequest: c.Next()
api_1 | /go/src/github.com/gin-gonic/gin/gin.go:352 (0x902dbd)
api_1 | (*Engine).ServeHTTP: engine.handleHTTPRequest(c)
api_1 | /usr/local/go/src/net/http/server.go:2774 (0x6dcc77)
api_1 | serverHandler.ServeHTTP: handler.ServeHTTP(rw, req)
api_1 | /usr/local/go/src/net/http/server.go:1878 (0x6d8860)
api_1 | (*conn).serve: serverHandler{c.server}.ServeHTTP(w, w.req)
api_1 | /usr/local/go/src/runtime/asm_amd64.s:1337 (0x45a090)
api_1 | goexit: BYTE $0x90 // NOP
api_1 |
api_1 | [GIN] 2019/07/31 - 06:29:58 | 500 | 123.5997ms | 172.18.0.1 | GET /api/articles

As alternative solution, I changed server side codes to create table.
_, err = db.Exec("CREATE DATABASE IF NOT EXISTS article;")
if err != nil {
panic(err)
}
_, err = db.Exec("use article;")
if err != nil {
panic(err)
}
_, err = db.Exec("CREATE TABLE IF NOT EXISTS `articles` (`id` int NOT NULL AUTO_INCREMENT PRIMARY KEY,uuid varchar(36), `title` VARCHAR(100) NOT NULL,`content` TEXT NOT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8;")
if err != nil {
panic(err)
}
_, err = db.Exec("create table IF NOT EXISTS images (id int AUTO_INCREMENT NOT NULL PRIMARY KEY, article_uuid varchar(36), image_name varchar(50)); ")
if err != nil {
panic(err)
}

Related

SQL - insert multiple line without double

I want to insert multiple lines into a database, but I inserted only one line or insert with missing line
My SQL table structure:
CREATE TABLE `test`
(
`ID` INT UNSIGNED NOT NULL AUTO_INCREMENT,
`time` datetime NOT NULL,
`nb_osuser` int NOT NULL,
`osuser_name` nvarchar(100)NOT NULL,
`program` nvarchar(100)NOT NULL,
`last_line` nvarchar(100)NOT NULL,
PRIMARY KEY (`ID`),
UNIQUE KEY (`time`)
) ENGINE=InnoDB;
Insert query:
#Connect to the database.
my $dbh = DBI->connect("DBI:mysql:database=database;host=IP",
"hostname", 'password',
{'RaiseError' => 1});
while (my ($user, $ref) = each %counts) {
while (my ($program, $count) = each %$ref) {
#print "$count OSUSER with session $user and with program $program\n";
print "time = $time, count = $count, user = $user, program = $program, last_line = $last_line\n";
$request ="'$time', '$count', '$user', '$program', $last_line";
my $sth = $dbh->prepare("REPLACE `test` (time, nb, os_name, program, last_line) VALUES($request);")
}
}
Data in my table:
ID USER TERMINAL SERVICE
---------- ------------------------- --------------- -------------------------
1 toto titi roro
2 toto titi roro
4 gigi gege fefe
I want:
+----+---------------------+-----------+-------------+----------------+-----------+
| ID | time | nb | os_name | program | last_line |
+----+---------------------+-----------+-------------+----------------+-----------+
| 15 | 2019-01-04 14:00:00| 33 | titi | roro | 109 |
| 16 | 2019-01-04 14:00:00| 9 | gege | fefe | 109 |
Thanks for any information

Error (1366, "Incorrect string value: '\\xE9 des ...' for column 'title' at row 29") occured in import csv to mysql

When I try to imprt data from csv file to mysql, an OperationalError occured:
OperationalError: (1366, "Incorrect string value: '\\xE9 des ...' for column 'title' at row 29")
My sql config is as below (version 5.7.22):
mysql> show variables like '%char%';
+--------------------------------------+----------------------------+
| Variable_name | Value |
+--------------------------------------+----------------------------+
| character_set_client | utf8 |
| character_set_connection | utf8 |
| character_set_database | utf8 |
| character_set_filesystem | binary |
| character_set_results | utf8 |
| character_set_server | utf8 |
| character_set_system | utf8 |
| character_sets_dir | /usr/share/mysql/charsets/ |
| validate_password_special_char_count | 1 |
+--------------------------------------+----------------------------+
9 rows in set (0.00 sec)
My schema is
"""
...: CREATE TABLE `Movies` (
...: `id` int(11) not null auto_increment,
...: `title` varchar(255) default '',
...: `genres` varchar(255) default '',
...: primary key(`id`)) engine=innodb charset=utf8mb4"""
The csv file is the movies.csv downloaded from ml-latest-small.zip .
To read the csv file, one function is written:
def import_data(tablename, filename):
with open(filename) as f:
reader = csv.reader(f)
keys = next(reader)
keys[0] = 'id' # change `movieId` in csv to `id`.
database.insertmany(tablename, keys, reader)
When I run import_data('Movies', 'movies.csv'), the error occured:
OperationalError: (1366, "Incorrect string value: '\\xE9 des ...' for column 'title' at row 29")
I checked row 29 in the movies.csv, accordingly:
movieId,title,genres
30,Shanghai Triad (Yao a yao yao dao waipo qiao) (1995),Crime|Drama
I Couldn't figure out the reason. Anyone having any ideas for this issue? Many thanks~

Retrieving connection status

Is there a way the status command be called on another connection. Something like:
status 231073
where the number is the ID from SHOW PROCESSLIST. In particular I'm interested in connection/client character sets.
EDIT:
By status I mean the command you call from the MySQL shell (not show status)
You could use the following to capture info of your connections, say, at the start of your app or at anytime. For instance max_allowed_packet could change during the client run lifecycle. So, in short, have in as part of your app startup.
Then you have collected info on your client connections, such as character set settings.
Schema:
DROP TABLE IF EXISTS connectionSnapshot;
CREATE TABLE connectionSnapshot
( -- based off of (more or less) mysql 5.6 show create table information_schema.PROCESSLIST;
-- 3 columns at least
-- TODO: compare to same structure on mysql 5.7
id INT AUTO_INCREMENT PRIMARY KEY,
connId BIGINT UNSIGNED NOT NULL,
user VARCHAR(16) NOT NULL,
host VARCHAR(64) NOT NULL,
login VARCHAR(100) NOT NULL,
-- the followed just dreamt up:
character_set_client VARCHAR(100) NOT NULL,
character_set_connection VARCHAR(100) NOT NULL,
character_set_results VARCHAR(100) NOT NULL,
collation_connection VARCHAR(100) NOT NULL,
lc_time_names VARCHAR(100) NOT NULL,
max_allowed_packet BIGINT NOT NULL,
time_zone VARCHAR(100) NOT NULL,
theWhen DATETIME NOT NULL
);
Stored Proc:
DROP PROCEDURE IF EXISTS uspConnectionSnapshotMe;
DELIMITER $$
CREATE PROCEDURE uspConnectionSnapshotMe()
BEGIN
DECLARE lconnId INT;
DECLARE luser VARCHAR(16);
DECLARE lhost VARCHAR(64);
SELECT connection_id() INTO lconnID;
SELECT USER,HOST into luser,lhost FROM information_schema.PROCESSLIST WHERE ID=lconnId;
INSERT connectionSnapshot (connId,user,host,login,character_set_client,character_set_connection,character_set_results,
collation_connection,lc_time_names,max_allowed_packet,time_zone,theWhen) VALUES
(lconnId,luser,lhost,current_user(),##session.character_set_client,##session.character_set_connection,
##session.character_set_results,##session.collation_connection,
##session.lc_time_names,##session.max_allowed_packet,##session.time_zone,now());
END$$
DELIMITER ;
-- ****************************************************************************************
Test:
call uspConnectionSnapshotMe();
Results:
select * from connectionSnapshot;
+----+--------+------+-----------------+----------------+----------------------+--------------------------+-----------------------+----------------------+---------------+--------------------+-----------+---------------------+
| id | connId | user | host | login | character_set_client | character_set_connection | character_set_results | collation_connection | lc_time_names | max_allowed_packet | time_zone | theWhen |
+----+--------+------+-----------------+----------------+----------------------+--------------------------+-----------------------+----------------------+---------------+--------------------+-----------+---------------------+
| 1 | 3825 | root | localhost:4660 | root#localhost | utf8 | utf8 | utf8 | utf8_general_ci | en_US | 4194304 | SYSTEM | 2016-09-08 02:40:18 |
| 2 | 37007 | root | localhost:52071 | root#localhost | utf8 | utf8 | cp850 | utf8_general_ci | en_US | 4194304 | SYSTEM | 2016-09-08 02:44:17 |
+----+--------+------+-----------------+----------------+----------------------+--------------------------+-----------------------+----------------------+---------------+--------------------+-----------+---------------------+

Mysql ignore foreign key errors

I have a BASH program that inserts into my DB somes lines.
Unfortunately, when MYSQL encounter an error(especially foreign key constraint errors), the entire SQL request stops.
Isn't there a way to skip the lines where the foreign key constraint doesn't apply, and keep adding the others?
insertVtigerInfos() {
$mysql -h $db_address -u $db_user -p$db_passwd $db_name -e "delete from $db_vtiger_name;load data local infile '$vtiger_temporary_file' REPLACE INTO TABLE $db_vtiger_name fields terminated by ';'"
}
----------------------démarrage---------------------------------
Tue May 6 16:03:13 CEST 2014
ERROR 1452 (23000) at line 1: Cannot add or update a child row: a foreign key constraint fails (`RH2QUALIF`.`info_vtiger`, CONSTRAINT `info_vtiger_ibfk_1` FOREIGN KEY (`nom_caisse`) REFERENCES `definition_ip_switch` (`nom_caisse`) ON DELETE CASCADE ON UPDATE CASCADE)
-------------------------fin------------------------------------
Tue May 6 16:03:14 CEST 2014
Updating thread with sample :
Table toward whom the foreign key constraint applies :
+-------------+------------+
| nom_caisse | quat_octet |
+-------------+------------+
| BCP | NULL |
| CEA | NULL |
Table having the foreign key constraint :
+-----------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+-------------+------+-----+---------+-------+
| code_site | varchar(32) | NO | PRI | NULL | |
| nom_caisse | varchar(32) | NO | MUL | NULL | |
| ip_routeur | varchar(32) | NO | | NULL | |
Data trying to be inserted :
131001M0;CEA;<ip>
131001G0;CEALS;<ip>
When mysql tries to insert the rows, the 1st is OK, the 2nd is not (because CEALS is not in the table which the foreign key is.
But mysql doesn't add ANY row because he found one not matching... I just want it to add the 1st row...
The only solution i found is to do that :
$mysql -f -h $db_address -u $db_user -p$db_passwd $db_name -e "SET foreign_key_checks=0;delete from $db_vtiger_name;load data local infile '$vtiger_temporary_file' REPLACE INTO TABLE $db_vtiger_name fields terminated by ';';SET foreign_key_checks=1;"
and then sending another request which delete the rows according to the foreign key.
delete from info_vtiger where NOT EXISTS ( select nom_caisse from definition_ip_switch where definition_ip_switch.nom_caisse=info_vtiger.nom_caisse ) ;
Unfortunately, the foreign key constraint is no more useful in this case.
posting it as a solution, if anyone have some other ways to do it...
If your requirement is just to insert new records but not replace to existing records then you can use as per below:
$mysql -h $db_address -u $db_user -p$db_passwd $db_name -e "delete from $db_vtiger_name;load data local infile '$vtiger_temporary_file' INSERT IGNORE INTO TABLE $db_vtiger_name fields terminated by ';'"

Linking a postgres table with a mysql tables fails at test

I am trying to reflect/mirror data from a Postgres table to a MySQL table. I found mysql_fwd and after connection everything to a localhost, I cannot enter data into the postgres FOREIGN TABLE. nor does data entered into MySQL, show up in postgres (backwards test)??
Data from MySQL does not reflect in postgres and if I test the other way (the correct direction) post_sync=# INSERT INTO mysql_sync (user_name, user_id) VALUES ('Joe', 3); ERROR: cannot change foreign table "mysql_sync"
*********** build process **************
CREATE SERVER mysql_svr
FOREIGN DATA WRAPPER mysql_fdw
OPTIONS (address '127.0.0.1', port '3306');
CREATE FOREIGN TABLE mysql_sync (id float4, user_name text, user_id integer)
SERVER mysql_svr OPTIONS ( database 'postgres_link', query 'SELECT * FROM mysql_sync');
CREATE USER MAPPING FOR mysqlLink SERVER mysql_svr OPTIONS (username 'some_dude', pass '');
***************** structure ************
Foreign table "public.mysql_sync"
Column | Type | Modifiers | FDW Options | Storage | Stats target | Description
-----------+---------+-----------+-------------+----------+--------------+-------------
id | real | | | plain | |
user_name | text | | | extended | |
user_id | integer | | | plain | |
Server: mysql_svr
FDW Options: (database 'postgres_link', query 'SELECT * FROM mysql_sync')
Has OIDs: no
******************** mySQL structure *************
mysql> desc mysql_sync;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 92
Current database: postgres_link
+-----------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| user_name | varchar(25) | NO | | NULL | |
| user_id | varchar(10) | NO | | NULL | |
+-----------+-------------+------+-----+---------+----------------+
3 rows in set (0.00 sec)
A whole bunch of different methods to convert, here -> http://wiki.postgresql.org/wiki/Converting_from_other_Databases_to_PostgreSQL