I'm trying to get TeamCity server running using docker-compose. Here's my compose file:
version: '3'
services:
db:
image: mysql
container_name: teamcity-db
restart: unless-stopped
env_file: .env
environment:
- MYSQL_DATABASE=teamcity
volumes:
- mysql:/var/lib/mysql
command: '--default-authentication-plugin=mysql_native_password'
teamcity:
depends_on:
- db
image: jetbrains/teamcity-server
container_name: teamcity
restart: unless-stopped
volumes:
- datadir:/data/teamcity_server/datadir
- logs:/opt/teamcity/logs
ports:
- "8111:8111"
volumes:
mysql:
datadir:
logs:
I've been successful getting wordpress set up using a very similar technique, and I can run phpMyAdmin and link it to the MySQL container and see the database, so its there.
When I browse to the teamcity web address, it shows me the initial setup screen as expected. I tell it to use MySQL and I put in 'root' as teh username and my MySQL root password. Teamcity then shows this:
I'm sure it's something simple but I just can't see what's wrong. Any ideas?
Solved! Here is my solution and some other learnings.
The problem was that I was telling TeamCity to use 'localhost' as the database server URL. This seems intuitive because all the services are on the same machine, but is incorrect. It is as if each container is its own host and so 'localhost' is specific to each container. 'localhost' inside a container refers to the container itself, not the host machine or any other container. So 'localhost' on the teamcity service refers to the teamcity server, not the database server, and that's why it couldn't connect.
The correct address for the database server based on my docker-compose.yml file is db (the service name of the database container). The service name becomes the host name for that container and docker resolves these as DNS names correctly within the composed group.
Also note: the default virtual network is created implicitly by docker-compose and allows all of the containers in the composed group to communicate with each other. The name of this network derives from the folder where the docker-compose.yml file is located (in my case ~/projects/teamcity) so I get a network called teamcity_default. All servers on this private vitual network are visible to each other with no further configuration needed.
The teamcity server container explicitly exposes port 8111 on the host's network interface, so it is the only container visible to the outside world. You do not need to (and probably should not) expose ports if you only need the servers to talk to each other. For example, the database server does not need to have a ports entry because it is automatically exposed on the private inter-container network. This is great for security because all the back-end services are hidden from the physical LAN and therefore the Internet.
Related
My TeamCity server seems to be using a broken URL for its built-in NuGet feed.
I'm running it in a docker container using the official JetBrains image. I'm not behind a reverse proxy. I have configured the "server URL" setting.
I can use the feed in Visual Studio using the full URL (unauthenticated guest access) and it all works great. It's adding packages from build artifacts, Visual Studio can pull them.
It's just that the TeamCity property that is supposed to contain the feed URL is broken, as shown in the screen shot. So my builds are failing like this:
/usr/share/dotnet/sdk/3.1.302/NuGet.targets(128,5): error : Unable to load the service index for source http://teamcity:8111/guestAuth/app/nuget/feed/TigraOss/TigraOSS/v3/index.json.
Those are internally generated and not something I've edited, so I'm a bit confuzzled. Any ideas on how to fix this? (I've tried restarting the server, obviously).
Update
I think this might be due to the fact that everything is running in docker containers. A bit later in the parameters screen (off the bottom of the screen shot above) is another line:
teamcity.serverUrl http://teamcity:8111
I think this is coming from my docker-compose.yml file:
agent:
image: jetbrains/teamcity-agent
container_name: teamcity-agent
restart: unless-stopped
privileged: true
user: "root"
environment:
- SERVER_URL=http://teamcity:8111
- AGENT_NAME=ubuntu-ovh-vps-tigra
- DOCKER_IN_DOCKER=start
volumes:
- agentconfig:/data/teamcity_agent/conf
- agentwork:/opt/buildagent/work
- agentsystem:/opt/buildagent/system
- agent1_volumes:/var/lib/docker
I tried changing the SERVER_URL value in my docker-compose.yml file and restarting the agent container, but it looks like once the agent config file is created, the value is sticky and I need to go in and hand-edit that.
Now I have the agent using the full FQDN of the server, so we'll see if that works.
I think this is caused by my complicated docker-in-docker build. I am running TeamCity server and the linux build agent in docker containers built with docker-compose. Here's my docker-compose.yml file with secrets removed:
version: '3'
services:
db:
image: mariadb
container_name: teamcity-db
restart: unless-stopped
env_file: .env
volumes:
- mariadb:/var/lib/mysql
command: --default-authentication-plugin=mysql_native_password
teamcity:
depends_on:
- db
image: jetbrains/teamcity-server
container_name: teamcity
restart: unless-stopped
environment:
- TEAMCITY_SERVER_MEM_OPTS="-Xmx750m"
volumes:
- datadir:/data/teamcity_server/datadir
- logs:/opt/teamcity/logs
ports:
- "8111:8111"
agent:
image: jetbrains/teamcity-agent
container_name: teamcity-agent
restart: unless-stopped
privileged: true
user: "root"
environment:
SERVER_URL: http://fully.qualified.name:8111
AGENT_NAME: my-agent-name
DOCKER_IN_DOCKER: start
volumes:
- agentconfig:/data/teamcity_agent/conf
- agentwork:/opt/buildagent/work
- agentsystem:/opt/buildagent/system
- agent1_volumes:/var/lib/docker
volumes:
mariadb:
datadir:
logs:
agentconfig:
agentwork:
agentsystem:
agent1_volumes:
networks:
default:
When I first created everything, I had the SERVER_URL variable set to `http://teamcity:8111". This works because Docker maps the host name to the service name, which is also 'teamcity' so that host is resolvable within the docker composition.
The problem comes when doing a build step inside yet another container.
I am building .NET Core and the .NET SDK is not installed on the machine,
so I have to run the build using the .NET Core SDK container.
The agent passes in the URL of the NuGet feeed, which is pointing to the docker service name, and the build container can't "see" that host name. I'm not sure why not. I tried passing in --network teamcity_default as a command line argument to docker run, but it says that network doesn't exist.
I found two ways to get things to work.
Edit the build step to use the FQDN of the nuget feed, and don't use the teamcity built-in parameter %teamcity.nuget.feed.guestAuth.feed-id.v3%. I don't like this solution much because it sets me up for a breakage in the future.
Find the docker volume where the teamcity agent config is stored. In my case, it was /var/lib/docker/volumes/teamcity_agentconfig/_data. Edit the buildAgent.properties file and set serverUrl=http\://fully.qualified.name\:8111. Then docker-compose restart agent. Then you can safely use %teamcity.nuget.feed.guestAuth.feed-id.v3% in containerized builds.
I haven't tested this, but I think you may be able to avoid all this in the first place by using a fully-qualified server name in the docker-compose.yml file. However you have to do this right from the start, because the moment you run docker-compose up the agent config filesystem is created and becomes permanent.
I have the following docker-compose.yml
version: '3'
services:
app:
build: .
network_mode: host
volumes:
- .:/usr/usr/src/app
db:
image: mysql/mysql-server:5.7
environment:
MYSQL_DATABASE: config_dev
MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
volumes:
- ./docker/images/config-dev-image/sql-scripts:/docker-entrypoint-initdb.d
restart: always
ports:
- "1200:3306"
My app service needs to connect to the db and using the documentation I tried to connect to using the service-name 'db' like so (from the app container)
mysql --hostname=db --port=3306 --user=root However, I get the error ERROR 2005 (HY000): Unknown MySQL server host 'db'
What am I doing wrong?
Your app container is running with network_mode: host. If it's using the host network then it can't use any of the Docker-specific network features; for example, it can't reach other containers by host name and it can't be reached by host name. For Docker networking purposes it's indistinguishable from a process running on the host.
Host networking isn't actually necessary for most of the cases I see suggested on SO, and you should see if your application works if you just remove that line. You might need to add ports: to make it accessible from outside.
If you really can't disable host networking, then you need to connect to the database the same way other processes running outside Docker network space would, via the other container's published ports. A host name of localhost should work (because you're in the context of the host) but you need the mapped port number --port=1200.
Add links configuration in app section
version: '3'
services:
app:
build: .
network_mode: host
volumes:
- .:/usr/usr/src/app
links:
- db
db:
image: mysql/mysql-server:5.7
environment:
MYSQL_DATABASE: config_dev
MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
volumes:
- ./docker/images/config-dev-image/sql-scripts:/docker-entrypoint-initdb.d
restart: always
ports:
- "1200:3306"
Because of network_mode: host your app container is effectively on a docker's private network like 192.168.65.1/24 while you db container is on a different private network created by docker-compose like 172.20.0.2/16. You can see this network being deleted when you run docker-compose down:
Removing network XXXX_default
where XXXX is your directory name.
If you were to remove network_mode: host from service app, both containers would be on the same private network and reachable by their service name.
$ docker inspect XXXX_default
"Containers": {
...
"Name": "app",
"IPv4Address": "172.21.0.3/16",
...
},
...
"Name": "db",
"IPv4Address": "172.21.0.2/16",
...
}
},
app container can access db on port 3306. No need to expose the port as 1200. As per docs:
Containers connected to the same user-defined bridge network
automatically expose all ports to each other, and no ports to the
outside world. This allows containerized applications to communicate
with each other easily, without accidentally opening access to the
outside world.
Docker noob alert. Hope this isn't a dumb question but I cannot seem to figure out what is going on. I am trying to create a docker-compose file which creates a mysql db with a mounted volume and a go webserver app that connects to the mysql db.
Here is my docker-compose file:
services:
db:
image: mysql:8.0.2
environment:
MYSQL_ROOT_PASSWORD: test
MYSQL_DATABASE: northernairport
ports:
- "3306:3306"
volumes:
- /data:/var/lib/mysql
web:
depends_on:
- db
build: .
ports:
- "8080:8080"
My go application can't seem to connect to my mysql db though, I thought the "depends_on" would ensure this was possible.
Error I get:
panic: dial tcp 127.0.0.1:3306: getsockopt: connection refused
Can anyone tell me what I am doing wrong here? Thanks.
The depends_on only controls the build and startup order for the services.
Your actual issue is more likely that you are using the wrong address from your web application to your database. I see that you have not defined any networks, so you are using the default network created for your application by docker-compose. This will publish each service by name on the default network's DNS.
So, your web application should probably be using db:3306 as the database address, not localhost:3306 or 127.0.0.1:3306 as indicated in the error message.
The ports part is used to map container ports with host in following format ports (HOST:CONTAINER). Which means that you are trying to access host's machine, configure web app to connect to db:3306 instead.
I have some trouble with setting my local git repository. I am new to docker, so the problem may be naive but I still can't find it.
So my idea is:
I wanted to create a container with gogs (gogs/gogs image) and connect it to mysql container.
To do so I have created docker-compose.yml file.
version: '3'
services:
db:
image: mysql
ports:
- "10023:3306"
environment:
- MYSQL_ROOT_PASSWORD=root!
ui:
image: phpmyadmin/phpmyadmin
ports:
- "8989:80"
links:
- db:mysql
repo:
image: gogs/gogs
ports:
- "10022:22"
- "10080:3000"
volumes:
- /tmp/gogs:/data gogs/gogs
links:
- db:mysql
I all put phpmyadmin in my setup, this way I can easily test if mysql is up and respond to other containers.
Sadly this environment does't work, when get to gogs install page on localhost:10080 and try to create a new repo, it says that tcp connection has been refused. This is the output of the error message:
dial tcp 127.0.0.1:10023: getsockopt: connection refused
This is strange, because I can access to mysql container through phpmyadmin. I also was able to create gogs database.
Do anybody had this issue before?
Don't use localhost or 127.0.0.1 from inside the container, use the service name as defined in your docker-compose.
dial tcp db:10023
docker-compose networking.
Thank a lot to gogsdoc_db_1. His/her answer is perfect. I will try to explain what I have done wrong, so maybe this would help new entry in docker such as myself.
As bluescores, have said. You shouldn't use localhost or 127.0.0.1 inside you docker container.
Why not?
Basically, when you use docker-compose it automatically create shared network for your app, which means you shouldn't use the published port.
In may case I must use port 3306, instead of 10023 to connect to my database container.
I should still use port 10023 if I want to connect to mysql from my local machine.
so my configuration now is
P.S.
Do not forget to create the database, before you install gogs
I use docker-compose with the following docker-compose.yml:
web_db:
image: mariadb:latest
restart: always
volumes:
- ./var/mysql:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: X
MYSQL_DATABASE: wordpress
MYSQL_USER: wordpress
MYSQL_PASSWORD: X
MYSQL_ROOT_PASSWORD: X
web_front:
image: nginx
restart: always
ports:
- 80:80
links:
- web_fpm
volumes:
- ./www:/var/www/html:rw
- ./etc/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
web_fpm:
build: ./PHP-FPM/
restart: always
links:
- web_db:mysql
volumes:
- ./www:/var/www/html
When configuring Wordpress (manually downloaded in nginx root directory), I'm asked for the address of the database. I've tried localhost, but it doesn't work.
PS: I know there are Wordpress images for Docker, but they have the full stack (nginx + PHP) and I don't like it, so don't propose them. ;)
Try 'web_db' instead of localhost
https://docs.docker.com/compose/compose-file/
Containers for the linked service will be reachable at a hostname identical to the alias, or the service name if no alias was specified.
You need to reference mysql because in your compose file you have linked your container to the web_db container and given it an alias of 'mysql'. The web_fpm container will have a record in it's hosts file for mysql which will point to the web_db container.
Alternatively, if you just want to get the ip to enter manually, just run docker inspect <web_db_container_id>
edit: You will want to put in the IP of the host machine along with port 3306 and here is why.
When starting a container with Docker it is started using an IP assigned from the Docker. These are all accessible to docker via the docker0 bridged network. Docker in turn makes those accessible to the host machine by bridging the docker0 NIC on the host machine with the host machines eth0/1 ect.
When you EXPOSE a port explicitly via docker run -td -p <some external port>:<some internal port> Docker opens that port up in the host IPTables so it is accessible on the local network. If you do not explicitly open the port but it is exposed in the container (the docker file will say EXPOSE 3306 for MySQL in its Dockerfile.) then the container is only available to the machine.
Now, when working with Docker containers it's important to know this because when you do a link with docker-compose you are only telling that container where another container is on the local docker network. This only helps if you have containers that start and are looking for a dependent container.
To summarize, when containers are started they are started with a Docker IP that is only used by Docker and bridged back to the host machine via docker0, and this is key. Docker isolates those IP and forces you to either open the ports on the machine to access via your local network.
Back to your question and why you need to pass it the host IP and port 3306.
You need to do this because you are making making a request on your network. If you put in localhost it will not work because the container is not really running on localhost it's running on some 172.17.42.x address so telling WordPress it's on localhost, it's not. It is however on at the host machines IP because the host machine is bridged to docker0. This is why you have to give it the host machines IP. Because you network can't resolve the 172.17 address since that is specific to Docker and specific to that machines containers. This is very fundamental to Docker networking and important to understand. See this article for more information.