How do I mount a persistent disk to a container volume? - google-compute-engine

I'm playing around with Google's managed VM feature and finding you can fairly easily create some interesting setups. However, I have yet to figure out whether it's possible to use persistent disks to mount a volume on the container, and it seems not having this feature limits the usefulness of managed VMs for stateful containers such as databases.
So the question is: how can I mount the persistent disk that Google creates for my Compute engine instance, to a container volume?

Attaching a persistent disk to a Google Compute Engine instance
Follow the official persistent-disk guide:
Create a disk
Attach to an instance during instance creation, or to a running instance
Use the tool /usr/share/google/safe_format_and_mount to mount the device file /dev/disk/by-id/google-...
As noted by Faizan, use docker run -v /mnt/persistent_disk:/container/target/path to include the volume in the docker container
Referencing a persistent disk in Google Container Engine
In this method, you specify the volume declaratively (after initializing it as mentioned above...) in the Replication Controller or Pod declaration. The following is a minimal excerpt of a replication controller JSON declaration. Note that the volume has to be declared read-only because no more than two instances may write to a persistent disk at one time.
{
"id": "<id>",
"kind": "ReplicationController",
"apiVersion": "v1beta1",
"desiredState": {
"replicas": 3,
"replicaSelector": {
"name": "<id>"
},
"podTemplate": {
"desiredState": {
"manifest": {
"version": "v1beta1",
"id": "<id>",
"containers": [
{
"name": "<id>",
"image": "<docker_image>",
"volumeMounts": [
{
"name": "persistent_disk",
"mountPath": "/pd",
"readOnly": true
}
],
...
}
],
"volumes": [
{
"name": "persistent_disk",
"source": {
"persistentDisk": {
"pdName": "<persistend_disk>",
"fsType": "ext4",
"readOnly": true
}
}
}
]
}
},
"labels": {
"name": "<id>"
}
}
},
"labels": {
"name": "<id>"
}
}

If your persistent disk is attached and mounted already to the instance, I believe you can use it as a data volume with your docker container. I was able to find docker documentation which explains the steps on how to manage data in containers.

Related

Failed to start minikube: Error while starting minikube. Error: X Exiting due to MK_USAGE: Container runtime must be set to "containerd" for rootless

I'm getting the error and I believe the way to solve it is by running: minikube start --container-runtime=containerd
but the extension seems to run minikube start. So how am I supposed to add the flag?
Here's the launch.json file
{
"configurations": [
{
"name": "Cloud Run: Run/Debug Locally",
"type": "cloudcode.cloudrun",
"request": "launch",
"build": {
"docker": {
"path": "Dockerfile"
}
},
"image": "dai",
"service": {
"name": "dai",
"containerPort": 8080,
"resources": {
"limits": {
"memory": "256Mi"
}
}
},
"target": {
"minikube": {}
},
"watch": true
}
]
}
Cloud Code for VS Code doesn't support such settings at the moment. But you can configure minikube to apply these settings with minikube config set.
The Cloud Run emulation creates a separate minikube profile called cloud-run-dev-internal. So you should be able to run the following:
minikube config set --profile cloud-run-dev-internal container-runtime containerd
You have to delete that minikube profile to cause the setting to take effect for your next launch:
minikube delete --profile cloud-run-dev-internal

Hashicorp Packer: vsphere-iso builder deploy multiple disks?

I'm trying to get Hashicorp's Packer (1.5.5) to deploy multiple disks for a specialized (USGCB compliant) template and I need it to have four separate disks. I'm using the vsphere-iso builder. The "disk_size" parameter deploys the root drive fine. I found the 'storage' parameter which I think is used to deploy the other disks...
storage ([]DiskConfig) - A collection of one or more disks to be provisioned along with the VM
But I can't seem to figure out the format that the data needs to be in - in the json file. I tried...
"storage": [
{
"storage[1]": "20673",
"storage[2]": "16384",
"storage[3]": "4096"
}
]
and
"storage": [
{
"20673",
"16384",
"4096"
}
]
but none of those worked.
the storage section contains just the additional disks
you need the following
"disk_controller_type": "pvscsi",
"disk_size": "20673",
"disk_thin_provisioned": true,
"storage": [
{
"disk_size": "16384",
"disk_thin_provisioned": true
},
{
"disk_size": "4096",
"disk_thin_provisioned": true
}
],

Substituting service url is arm template

I have an ARM template that deploys API's to an API Management instance
Here is an example of one API
{
"properties": {
"authenticationSettings": {
"subscriptionKeyRequired": false
},
"subscriptionKeyParameterNames": {
"header": "Ocp-Apim-Subscription-Key",
"query": "subscription-key"
},
"apiRevision": "1",
"isCurrent": true,
"subscriptionRequired": true,
"displayName": "DDD.CRM.PostLeadRequest",
"serviceUrl": "https://test1/api/FuncCreateLead?code=XXXXXXXXXX",
"path": "CRMAPI/PostLeadRequest",
"protocols": [
"https"
]
},
"name": "[concat(variables('ApimServiceName'), '/mms-crm-postleadrequest')]",
"type": "Microsoft.ApiManagement/service/apis",
"apiVersion": "2019-01-01",
"dependsOn": []
}
When I am deploying this to different environments I would like to be able to substitute the service url depending on the environment. I'm wondering the best approach?
Can I read in a config file or something like that?
At the time of deployment I have a variable that tells me the environment so I can base decisions on that. Just not sure the best way to do it
See about ARM template parameters: https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-authoring-templates#parameters They can be specified in a separate file. So you will have single template, but environment specific parameter files.

Deploy dockerized mysql container to edge device

I have built a docker application using docker-compose which has mysql involved in it.I have pushed those containers to azure and wanted to deploy it in an edge device using Azure IoT Edge. For this i used docker application container and mysql container to deploy in edge device, Application is running but mysql is not running at edge device after deployment.
Here is the container create options that i have given for mysql module
Is it because as i am using the root as User? Which is refusing connection with different client.
{
"Env": [
"ACCEPT_EULA=Y",
"MSSQL_ROOT_PASSWORD=root"
],
"HostConfig": {
"PortBindings": {
"13306/tcp": [
{
"HostPort": "13306"
}
],
"32000/tcp": [
{
"HostPort": "32000"
}
]
},
"Mounts": [
{
"Type": "volume",
"Source": "sqlVolume",
"Target": "/var/lib/mysql"
}
]
}

Cluster communication and firewalls in Google Container Engine

I'm trying to set up the following environment on Google Cloud and have 3 major problems with it:
Database Cluster
3 nodes
one port open to world, a few ports open to the compute cluster
Compute Cluster
- 5 nodes
- communicated with the database cluster
- two ports open to the world
- runs Docker containers
a) The database cluster runs fine, I have the configuration port open to world, but I don't know how to limit the other ports to only the compute cluster?
I managed to get the first Pod and Replication-Controller running on the compute cluster and created a service to open the container to the world:
controller:
{
"id": "api-controller",
"kind": "ReplicationController",
"apiVersion": "v1beta1",
"desiredState": {
"replicas": 2,
"replicaSelector": {
"name": "api"
},
"podTemplate": {
"desiredState": {
"manifest": {
"version": "v1beta1",
"id": "apiController",
"containers": [{
"name": "api",
"image": "gcr.io/my/api",
"ports": [{
"name": "api",
"containerPort": 3000
}]
}]
}
},
"labels": {
"name": "api"
}
}
}
}
service:
{
"id": "api-service",
"kind": "Service",
"apiVersion": "v1beta1",
"selector": {
"name": "api"
},
"containerPort": "api",
"protocol": "TCP",
"port": 80,
"selector": { "name": "api" },
"createExternalLoadBalancer": true
}
b) The container exposes port 3000, the service port 80. Where's the connection between the two?
The firewall works with labels. I want 4-5 different pods running in my compute cluster with 2 of them having open ports to the world. There can be 2 or more containers running on the same instance. The labels however are specific to the nodes, not the containers.
c) Do I expose all nodes with the same firewall configuration? I can't assign labels to containers, so not sure how to expose the api service for example?
I'll try my best to answer all of your questions as best I can.
First off, you will want to upgrade to using v1 of the Kubernetes API because v1beta1 and v1beta3 will no longer be available after Aug. 5th:
https://cloud.google.com/container-engine/docs/v1-upgrade
Also, Use YAML. It's so much less verbose ;)
--
Now on to the questions you asked:
a) I'm not sure I completely understand what you are asking here but it sounds like running the services in the same cluster (with resource limits) would be way easier than trying to deal with cross cluster networking.
b) You need to specify a targetPort so that the service knows what port to use on the container. This should match port 3000 that you have in your resource controller. See the docs for more info.
{
"kind": "Service",
"apiVersion": "v1",
"metadata: {
"labels": [{
"name": "api-service"
}],
},
"spec": {
"selector": {
"name": "api"
},
"ports": [{
"port": 80,
"targetPort": 3000
}]
"type": "LoadBalancer"
}
}
c) Yes. In Kubernetes the kube-proxy accepts traffic on any node and routes it to the appropriate node or local pod. You don't need to worry about mapping the load balancer to, or writing firewall rules for those specific nodes that happen to be running your pods (it could actually change if you do a rolling update!). kube-proxy will route traffic to the right place even if your service is not running on that node.