I've a kubernetes cronjob manifest file.In that file I've defined enviornment variables.I'm generating yaml using a shell script but while using the yaml using kubectl create -f. I'm getting the following validation error
error validating "cron.yaml": error validating data: [ValidationError(CronJob.spec.jobTemplate.spec.template.spec.containers[0].envFrom[0].configMapRef): invalid type for io.k8s.api.core.v1.ConfigMapEnvSource: got "array", expected "map".
Can anyone suggest me how to resolve this?
You have a mistake in the syntax.
There are two approaches, using valueFrom for individual values or envFrom for multiple values.
valueFrom is used inside the env attribute.valueFrom will inject the value of a a key from the referenced configMap.
spec:
template:
spec:
containers:
- name: ad-sync
image: foo.azurecr.io/foobar/ad-sync
command: ["dotnet", "AdSyncService.dll"]
args: []
env:
- name: AdSyncService
valueFrom:
configMapKeyRef:
name: ad-sync-service-configmap
key: log_level
envFrom is used direct inside the container attribute.envFrom will inject All configMap keys as environment variables
spec:
template:
spec:
containers:
- name: ad-sync
image: foo.azurecr.io/foobar/ad-sync
command: ["dotnet", "AdSyncService.dll"]
envFrom:
- configMapRef:
name: ad-sync-service-configmap
Related
I am on on openshift 4.6. I want to pass parameters when to a yaml file so i tried the below code but it threw an error
oc apply -f "./ETCD Backup/etcd_backup_cronjob.yaml" --param master-node = oc get nodes -o name | grep "master-0" | cut -d'/' -f2
Error: unknown flag --param
What you most likely want to use is OpenShift Templates. Using Templates you can have variables in your YAML files and then change them using oc process.
So your YAML would look like so:
kind: Template
apiVersion: v1
metadata:
name: my-template
objects:
- apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: pi
spec:
schedule: "*/1 * * * *"
concurrencyPolicy: "Replace"
startingDeadlineSeconds: 200
suspend: true
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 1
jobTemplate:
spec:
template:
metadata:
labels:
parent: "cronjobpi"
spec:
containers:
- name: pi
image: perl
command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(${{DIGITS}})"]
restartPolicy: OnFailure
parameters:
- name: DIGITS
displayName: Number of digits
description: Digits to compute
value: 200
required: true
Then you can use oc process like so:
oc process my-template.yml --param=DIGITS=300 | oc apply -f -
I'm writing AWS CloudFormation template (using yaml) which creates AWS Service Catalog Product.
I'm getting the template for the product using parameter S3FilePath which has a value like the above path: https://bucket.s3-eu-west-1.amazonaws.com/template.yml.
The URL to the file needs to be send in a JSON format as shown here (this example works):
Resources:
Type: AWS::ServiceCatalog::CloudFormationProduct
Properties:
Description: Example Product
Distributor: xyz
Name: ExampleProduct
Owner: xyz
ProvisioningArtifactParameters:
- Description: Example Product
Info: { "LoadTemplateFromURL": "https://bucket.s3-eu-west-1.amazonaws.com/template.yml" }
Name: Version1
I tried to replace the URL using !Sub and !Ref as shown below:
Parameters:
S3FilePath:
Type: String
Description: file name
Resources:
Type: AWS::ServiceCatalog::CloudFormationProduct
Properties:
Description: Example Product
Distributor: xyz
Name: ExampleProduct
Owner: xyz
ProvisioningArtifactParameters:
- Description: Example Product
Info: !Sub
- '{ "LoadTemplateFromURL": "${FILEPATH}" }'
- {FILEPATH: !Ref S3FilePath}
Name: Version1
But the CloudFormation stack fails with the error: "invalid input".
I guess I am building the JSON in a wrong way, I tried to use \ before each ' " ' but it didn't help either and I couldn't find an example which explain how to build this correctly. There is no problem with the S3FilePath parameter.
Can you please advice how to use the !Sub and !Ref correctly to build the JSON? Thanks.
Here is an example: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-sub.html#w2ab1c25c28c59c11
Despite the documentation saying the Info parameter is JSON, the example shows just a name/value pair (Map): https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-servicecatalog-cloudformationproduct-provisioningartifactproperties.html
Try formatting your string as
Info: !Sub
- "LoadTemplateFromURL": "${FILEPATH}"
- {FILEPATH: !Ref S3FilePath}
You can reference any Parameters or LogicalResourceId directly inside a !Sub like so:
ProvisioningArtifactParameters:
- Description: Example Product
Info: !Sub '{ "LoadTemplateFromURL": "${S3FilePath}" }'
Name: Version1
This should work totally fine. The way you were doing substitutions is useful when you want to use conditions and/or mapping inside a !Sub.
I think it should be simply:
ProvisioningArtifactParameters:
- Description: Example Product
Info:
LoadTemplateFromURL: !Ref S3FilePath
Name: Version1
This is at least what I have in my own AWS::ServiceCatalog::CloudFormationProduct templates.
ProvisioningArtifactParameters:
- DisableTemplateValidation: false
Info:
LoadTemplateFromURL: !Ref S3FilePath
I want to pass parameter dynamically while patching any deployment config
oc patch dc/action-msa -p "$(cat msa-patch.yml)" --param service_account=msa-service-account
spec:
template:
spec:
serviceAccountName: ${service_account}
restartPolicy: "Always"
initContainers:
- name: vault-init
image: ${init_container_image}
imagePullPolicy: Always
containers:
- name: ${SERVICE_NAME}-java-service
image: ${main_container_image}
Is there any option or way to pass service_account, init_container_image and service_name dynamically while patching using openshift oc?
You would need template layer for this solution such as Kustomize, HELM and etc.
Or you can use environment file as a source before deploying your yaml files something like below
Your deployment.yaml looks like this:
spec:
template:
spec:
serviceAccountName: {{service_account}}
restartPolicy: "Always"
initContainers:
- name: vault-init
image: {{init_container_image}}
imagePullPolicy: Always
containers:
- name: {{SERVICE_NAME}}
image: {{main_container_image}}
your env.file looks like this:
service_account="some_account"
init_container_image="some_image"
SERVICE_NAME="service_name"
Then run
oc patch dc/action-msa -p \
"$(source env.file && cat deployment.yml | \
sed "s/{{service_account}}/service_account/g"| \
sed "s/{{init_container_image}}/init_container_image/g"| \
sed "s/{{SERVICE_NAME}}/SERVICE_NAME/g")" --param service_account=msa-service-account
Hope it helps
I have two Openshift environments that I publish to from a Gitlab pipeline. Let's call the first one DEV and it runs Openshift v3.7.1+c2ce2c0-1. The second one is INT and runs v3.6.1+008f2d5. Recently DEV got upgraded from 3.6.1 to 3.7.1 and after that I noticed a strange change in the behavior of redeployment triggers.
In short, what I see is that existing deployments in the DEV environment are triggered to redeploy with an "Image changed" message when an unchanged deployment config template is applied and while the Docker image also remains unchanged. This means that for example a MongoDB or Jenkins deployment recreates all containers and loses all data with every run of the CI pipeline.
Yes, there is the theoretically the possibility to use persistent volumes but the Openshift installations are not under my control. The point here is that redeployments happen when neither image nor configuration changes.
The command that I am using from Gitlab is this:
oc process -f openshift-mongodb-ephemeral.yml -v MONGODB_DATABASE=mydb -v DATABASE_SERVICE_NAME=mongodb -l template=northbound-mongodb | oc apply -f -
Here is one of the deployment config templates:
apiVersion: v1
kind: Template
labels:
template: mongodb-ephemeral-template
metadata:
annotations:
description: |-
MongoDB database service, without persistent storage. For more information about using this template, including OpenShift considerations, see https://github.com/sclorg/mongodb-container/blob/master/3.2/README.md.
WARNING: Any data stored will be lost upon pod destruction. Only use this template for testing
iconClass: icon-mongodb
openshift.io/display-name: MongoDB (Ephemeral)
tags: database,mongodb
creationTimestamp: 2017-03-14T11:25:13Z
name: mongodb-ephemeral
resourceVersion: "483"
selfLink: /oapi/v1/namespaces/openshift/templates/mongodb-ephemeral
uid: e41b7f8e-08a8-11e7-9120-000d3a266151
objects:
- apiVersion: v1
kind: Service
metadata:
creationTimestamp: null
name: ${DATABASE_SERVICE_NAME}
spec:
ports:
- name: mongo
nodePort: 0
port: 27017
protocol: TCP
targetPort: 27017
selector:
name: ${DATABASE_SERVICE_NAME}
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
- apiVersion: v1
kind: DeploymentConfig
metadata:
creationTimestamp: null
name: ${DATABASE_SERVICE_NAME}
spec:
replicas: 1
selector:
name: ${DATABASE_SERVICE_NAME}
strategy:
type: Recreate
template:
metadata:
creationTimestamp: null
labels:
name: ${DATABASE_SERVICE_NAME}
spec:
containers:
- capabilities: {}
env:
- name: MONGODB_USER
value: ${MONGODB_USER}
- name: MONGODB_PASSWORD
valueFrom:
secretKeyRef:
key: database-password
name: myproject-secrets
- name: MONGODB_ADMIN_PASSWORD
valueFrom:
secretKeyRef:
key: database-admin-password
name: myproject-secrets
- name: MONGODB_DATABASE
value: ${MONGODB_DATABASE}
image: ' '
imagePullPolicy: IfNotPresent
livenessProbe:
initialDelaySeconds: 30
tcpSocket:
port: 27017
timeoutSeconds: 1
name: mongodb
ports:
- containerPort: 27017
protocol: TCP
readinessProbe:
exec:
command:
- /bin/sh
- -i
- -c
- mongo 127.0.0.1:27017/$MONGODB_DATABASE -u $MONGODB_USER -p $MONGODB_PASSWORD
--eval="quit()"
initialDelaySeconds: 3
timeoutSeconds: 1
resources:
limits:
memory: ${MEMORY_LIMIT}
securityContext:
capabilities: {}
privileged: false
terminationMessagePath: /dev/termination-log
volumeMounts:
- mountPath: /var/lib/mongodb/data
name: ${DATABASE_SERVICE_NAME}-data
dnsPolicy: ClusterFirst
restartPolicy: Always
volumes:
- emptyDir:
medium: ""
name: ${DATABASE_SERVICE_NAME}-data
triggers:
- imageChangeParams:
automatic: true
containerNames:
- mongodb
from:
kind: ImageStreamTag
name: mongodb:${MONGODB_VERSION}
namespace: ${NAMESPACE}
lastTriggeredImage: ""
type: ImageChange
- type: ConfigChange
status: {}
parameters:
- description: Maximum amount of memory the container can use.
displayName: Memory Limit
name: MEMORY_LIMIT
required: true
value: 512Mi
- description: The OpenShift Namespace where the ImageStream resides.
displayName: Namespace
name: NAMESPACE
value: openshift
- description: The name of the OpenShift Service exposed for the database.
displayName: Database Service Name
name: DATABASE_SERVICE_NAME
required: true
value: mongodb
- description: Name of the MongoDB database accessed.
displayName: MongoDB Database Name
name: MONGODB_DATABASE
required: true
value: sampledb
- description: Version of MongoDB image to be used (2.4, 2.6, 3.2 or latest).
displayName: Version of MongoDB Image
name: MONGODB_VERSION
required: true
value: "3.2"
- description: Name of user to access MongoDB.
displayName: MongoDB user
name: MONGODB_USER
required: true
value: "mongouser"
The oc process and oc apply commands get executed with each execution of the Gitlab CI pipeline. The pipeline triggers whenever someone merges into e.g. the develop branch. I would like to keep this behavior because this guarantees if someone changes the configuration of MongoDB, Jenkins, etc. those get updated and redeployed automatically (in which case a loss of data is acceptable).
Does anybody know what change in OS could have prompted this change in behavior and how to achieve the old behavior again?
Credits go to Graham Dumpleton who answered my question, albeit through a comment. Since I did not hear back from him I am leaving this answer myself for completeness' sake.
After removing the entries
creationTimestamp
lastTriggeredImage
status
image
from the deployment config template, the unwanted redeployments stopped. I do not know which entry specifically started triggering the behavior in Openshift 3.7.1 but in any case, these four entries are information generated during an Openshift deployment process and should not be in a template to begin with.
I would like to provide an environment variable to an OpenShift pod and within that variable reference other environment variables defined in the container.
For example, I define an environment variable called JAVA_CMD_LINE in OpenShift and set it to:
$HEAP_SETTING -Djavax.net.ssl.trustStore=/var/.keystore/cacerts -jar abc.jar
Where $HEAP_SETTING is set to -XMX=1G when the container starts.
In my container, there is a startup script that looks like:
java $JAVA_CMD_LINE
What I would expect is that then the container runs, the following is executed:
java -XMX=1G -Djavax.net.ssl.trustStore=/var/.keystore/cacerts -jar abc.jar
But instead what I see is:
java '$HEAP_SETTING' -Djavax.net.ssl.trustStore=/var/.keystore/cacerts -jar abc.jar
How do I provide the variable?
Update: Adding details from the YML file.
spec:
containers:
- env:
- name: OPENSHIFT_ENABLE_OAUTH
value: 'true'
- name: OPENSHIFT_ENABLE_REDIRECT_PROMPT
value: 'true'
- name: KUBERNETES_MASTER
value: 'https://kubernetes.default:443'
- name: KUBERNETES_TRUST_CERTIFICATES
value: 'true'
- name: JAVA_CMD_LINE
value: >-
-Djavax.net.ssl.trustStore=/var/cert/.keystore/cacerts
-Dfile.encoding=UTF8
$HEAP_SETTING
Update 2 - The error that I see:
+ exec java -Djavax.net.ssl.trustStore=/var/jenk-cert/.keystore/cacerts -Djavax.net.ssl.trustStorePassword=changeit -Dfile.encoding=UTF8 '$(HEAP_SETTING)' -Duser.home=/var/lib/jenkins -Djavamelody.application-name=JENKINS -jar /usr/lib/jenkins/jenkins.war
Picked up JAVA_TOOL_OPTIONS: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -Dsun.zip.disableMemoryMapping=true
Error: Could not find or load main class $(HEAP_SETTING)
Try using:
spec:
containers:
- env:
- name: OPENSHIFT_ENABLE_OAUTH
value: 'true'
- name: OPENSHIFT_ENABLE_REDIRECT_PROMPT
value: 'true'
- name: KUBERNETES_MASTER
value: 'https://kubernetes.default:443'
- name: KUBERNETES_TRUST_CERTIFICATES
value: 'true'
- name: JAVA_CMD_LINE
value: >-
-Djavax.net.ssl.trustStore=/var/cert/.keystore/cacerts
-Dfile.encoding=UTF8
$(HEAP_SETTING)
Any time you are setting the value of an environment variable, if you need to compose the value from other environment variables that are already being set, you can use $(<VARNAME>) in the value.
IOW, use $(HEAP_SETTING) and not just $HEAP_SETTING.
UPDATE 1
Actually this will not work. This is because HEAP_SETTING is not in the set of environment variables you are setting via the deployment config, so it will pass the literal value $(HEAP_SETTING). This can't be used where the environment variable you are trying to use is populated by startup code in the image.