Is it possible to have several flume's agents (sinks) under the same configuration file (agent.conf)?
I think so. It is a matter of include all the per-sinks configuration in the same agent.conf file. There is an example here.
The preferred way for FIWARE is using Dockers. So, let's imagine we need a Cygnus and we want the data to be "sinked" to MongoDB and MySQL.
A good practice would consist of making a Docker-compose file in order to build the application, but in this case, I'll show how to deploy all dockers needed separately.
We want to deploy a MySQL so Cygnus can store data in it. We can do it this way:
sudo docker run --name mysql_showcases \
-e MYSQL_ROOT_PASSWORD=root \
-e MYSQL_DATABASE=dbcygnus \
-e MYSQL_USER=cygnus \
-e MYSQL_PASSWORD=cygnus \
-e MYSQL_ROOT_HOST='%' \
-p 3306:3306 -it -v /data/mysql:/var/lib/mysql -d -h mysql mysql/mysql-server:5.5
We want to deploy a MongoDB so Cygnus can also store data in it. We can do it this way:
sudo docker run --name mongo_showcases -v /data/mongodb:/data/db -d \
-h mongo mongo:3.6
Finally, we can deploy Cygnus using a Docker linked with both previous dockers:
docker run -d --name cygnus_showcases --link mysql_showcases --link mongo_showcases \
-p 8081:8081 -p 5050:5050 \
-e CYGNUS_MYSQL_HOST=mysql_showcases -e CYGNUS_MYSQL_PORT=3306 \
-e CYGNUS_MYSQL_USER=root -e CYGNUS_MYSQL_PASS=root \
-e CYGNUS_MONGO_HOSTS=mongo_showcases:27017 \
fiware/cygnus-ngsi
So, we've deployed a Docker, using Cygnus which will store data in a MongoDB and a MySQL database. We can also provide more "variables" to configure other sinks to where to store data in.
Related
I'm starting to use docker implement mysql in our environment. But I have a little bit confuse about it.
1. I have tried to use command, it's working
sudo docker run --name mysql5.7 --restart always --privileged=true -p 4306:3006 -v /Users/user/mysql/config/mysqld.cnf:/etc/mysql/mysql.conf.d/mysqld.cnf -v /Users/user/mysql/data:/var/lib/mysql -e MYSQL_USER=“usr” -e MYSQL_PASSWORD=“1234” -e MYSQL_ROOT_PASSWORD=“1234” -d mysql:5.7
But follow docker document, they suggest use volume to persist data. So I tried crate a volume first docker volume -d create local mysql_v
try to link mysql to volume mysql_v, but I don't know how to do it and what is different with step 1.
anyone can suggest it ~?
Like
docker run --name mysql5.7 --restart always -p 4306:3006 \
-v /Users/user/mysql/config/mysqld.cnf:/etc/mysql/mysql.conf.d/mysqld.cnf \
-v mysql_v:/var/lib/mysql \
-e MYSQL_USER=“usr” -e MYSQL_PASSWORD=“1234” \
-e MYSQL_ROOT_PASSWORD=“1234” -d mysql:5.7
Note, privileged removed, that's just asking for trouble
Ref: official documents
I need to start a mysql docker container with an initial dump file or create a mysql docker image with initial data.
I don't need to start docker container first and then execute with the dump file in separate commands. I need to run it using a single command.
So assuming the data contains your sql dump file:
docker run -v "$PWD/data":/docker-entrypoint-initdb.d \
--user 1000:1000 \
--name some-mysql \
-e MYSQL_ROOT_PASSWORD=my-secret-pw \
-d \
mysql:tag
ref: hub
Hello i have created mysql image and with this command
docker run --name db-mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:latest --> Run container with my sql
docker pull mysql --> create image with mysql
docker run --name db_mysql-e MYSQL_ROOT_PASSWORD=1234 -e MYSQL_DATABASE=mami -p 3306:3306 -d mysql
i execute it but after that i don't know what to do and how to make a DB in this container and the job for the backup
If someone can help me with step by step what to do
You could use the cron service from your host system to run the following command as described in the documentation for the mysql docker image:
crontab example for running the command every night at 2:00 am:
00 02 * * * /usr/bin/docker exec db-mysql sh -c 'exec mysqldump --all-databases -uroot -p"my-secret-pw"' > /some/path/on/your/host/all-databases.sql
Alternatively you could run another container designed just for this task such as deitch/mysql-backup:
docker run --name db-mysql -d \
-e MYSQL_ROOT_PASSWORD=my-secret-pw \
-e MYSQL_USER=my-user \
-e MYSQL_PASSWORD=my-user-password \
-e MYSQL_DATABASE=my-db \
mysql:latest
docker run -d --restart=always \
--name=db-backup \
-e DB_DUMP_BEGIN=0200 \
-e DB_SERVER=db-mysql \
-e DB_USER=my-user \
-e DB_PASS=my-user-password \
-e DB_NAMES=my-db \
-e DB_DUMP_TARGET=/db \
-v /somewhere/on/your/host/:/db \
databack/mysql-backup
You also need to make sure the /somewhere/on/your/host/ folder is writable by users of group 1005:
sudo chgrp 1005 /somewhere/on/your/host/
sudo chmod g+rwX /somewhere/on/your/host/
But this container must have a mean to connect to your db-mysql container. For that you create a docker network and connect both containers to it:
docker network create mysql-backup-net
docker network connect mysql-backup-net db-backup
docker network connect mysql-backup-net db-mysql
I tried to run mysql 5.6 from docker like this:
docker run --name mysqlxx -e MYSQL_ROOT_PASSWORD=xxx \
--character-set-server=utf8mb4 \
--collation-server=utf8mb4_unicode_ci \
-d mysql:5.6
but got the following error
unknown flag: --character-set-server
anyone knows what happens? is it a bug of mysql 5.6 Dockerfile?
the image is pulled from https://hub.docker.com/_/mysql
What you have written means that you pass --character-set-server and --collation-server as arguments to docker, and NOT to mysql.
As soon as these flags are related to MySQL, you have to pass them to MySQL service, not docker. Command line for container starts right after image name (mysql:5.6). That will look somehow like:
docker run --name mysqlxx -e MYSQL_ROOT_PASSWORD=xxx -d mysql:5.6 mysql --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
If you can supply them as environment, better use this approach, but you would have to prepend -e for each environment variable. Sorry, I have no idea if MySQL accepts such parameters from environment
as #grapes said, the arguments should be put after image name. below command works.
docker run --name mysqlxx -p 3336:3306 \
-e MYSQL_ROOT_PASSWORD=xxx -d mysql:5.6 \
--character-set-server=utf8mb4 \
--collation-server=utf8mb4_unicode_ci
How can i mysqldump from running container on https://hub.docker.com/_/mariadb/ ?
I cant find any useful documentation or data?
Any method for backup and restore database.
This is my my continaer run command :
docker run --name myaapp-mariadb -v /databases/maria:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=password -d mariadb:10
If we assume you created the mariadb server container this way:
docker run --name some-mariadb -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mariadb:latest
Then you access it from another client container:
docker run -it --link some-mariadb:mysql \
--rm mariadb:latest \
sh -c 'exec mysqldump -h"$MYSQL_PORT_3306_TCP_ADDR" -P"$MYSQL_PORT_3306_TCP_PORT" -uroot -p"$MYSQL_ENV_MYSQL_ROOT_PASSWORD" database_name' > database_name_dump.sql
There's lots more helpful usage tip in the mysql official image page.
Accepted answer stands accepted & correct in all its sense. Adding, this for the case where we have mapped the database to an external volume.
So, for example if the container was created using below command
docker run --name mysqldb -p 3306:3306 -e MYSQL_ROOT_PASSWORD=password -v /dir_path_on_your_machine/mysql-data:/var/lib/mysql -d mariadb:latest
then, we can execute the below command from cmd line or terminal
docker exec mysqldb mysqldump --user=root --password=password dbname > /dir_path_on_your_machine/mysql-data/dump/db.sql
However, the dump created using above commands will not dump stored procedures, functions and events. We would need extra params with the in order to do that
--triggers Dump triggers for each dumped table.
--routines Dump stored routines (functions and procedures).
--events Dump events.
So, we can modify our command to include the above params for desired result.
Sample update command
docker exec mysqldb mysqldump --routines --triggers --user=root --password=password dbname > /dir_path_on_your_machine/mysql-data/dump/db1.sql
In case, you encounter any import related error ,, check if this helps.