We are using kubebuilder to build our custom controller, the problem is we are not able to parse raw data as it comes empty when you apply the file.
I’ve created very minimal example which describe the issue.
apiVersion: mygroup.test.com/v1alpha1
kind: Rawtest
metadata:
name: rawtest-sample
spec:
system:
type: test
provider:
type: aws
infrastructureConfig:
kind: InfrastructureConfig
apiVersion: v1alpha1
networks:
vpc:
cidr: aaa
zones:
- name: abc
internal: 123
workers:
- name: myworker
machine:
type: "mt"
image:
name: name1
version: "2"
maximum: 2
minimum: 1
maxUnavailable: 0
volume:
type: a1
size: 20Gi
zones:
- zone1
In runtime I was able to get the the spec.system.type value=test and spec.system.provider.type value="aws", however I wasn’t able to get all the data under the infrastructureConfig: (line 10) any idea how can I overcome this ?
I’ve created this very simple project to demonstrate the issue ,
See the api/type folder, after getting the reconcile object (after apply the config/sample/ file ,you see that the infrastructureconfig and all related data are
https://github.com/JennyMet/
Here is the code which is trying to read the raw value
https://github.com/JennyMet/kuberaw/blob/master/controllers/rawtest_controller.go#L57
&rawtest should contain all the data
please see the type
https://github.com/JennyMet/kuberaw/blob/master/api/v1alpha1/rawtest_types.go#L32
raw type
https://github.com/gardener/gardener/blob/bf32324d9d1a366d8a0a7514956dc39c2f22f7b7/pkg/apis/core/v1beta1/types_shoot.go#L945
https://github.com/gardener/gardener/blob/bf32324d9d1a366d8a0a7514956dc39c2f22f7b7/pkg/apis/core/types_shoot.go#L774
https://github.com/gardener/gardener/blob/bf32324d9d1a366d8a0a7514956dc39c2f22f7b7/vendor/k8s.io/apimachinery/pkg/runtime/types.go#L94:6
I need a way to make it work in the kubebuilder, as while I apply the file I dont get the values in debug ...
debug pic
if something is missing please let me know.
I did the following
visit a site such as https://yaml.to-go.online/ convert the yaml to a go Struct
make a struct
t := AutoGenerated{}
I assume that the yaml data is loaded into memory as []byte
using "gopkg.in/yaml.v2", Unmarshall the yaml to the struct
err := yaml.Unmarshal([]byte(data), &t)
the infrastructureconfig field is available as t.Spec.System.Provider.InfrastructureConfig
see https://play.golang.org/p/syx8v7gAmDH
Related
I am trying to create a secret using JSON file content and stringData like below but giving some error which I am not able to identify after multiple tries.
apiVersion: v1
kind: Secret
metadata:
name: image-secret
type: Opaque
stringData:
creds: _json_key:{"type": "service_account","project_id": "xyz","private_key_id": "9b0eb25b41ae9161123dbfh56mgj","private_key": "-----BEGIN PRIVATE KEY-----\nmch0iiFz1DAdM8vQTXiETI+3gvSnknXQ0M5WmkA1dkiJgyhe3r8tpeb42jo4FCd\nbHLf9eeIql8TKEm9BAk+qnQZq8FykWEnQLuU7APrFNZ0qtYP8t1Y7HSGpdVmmCyK\nykJAGznKaiEf9SJiNy8HqJy1kOhajn1fL3CdcShWcY793qRLyeFyrIZ\n6lfnjSE9IW5iEOBmxEpXf5Q=\n-----END PRIVATE KEY-----\n","client_email": "argocd-image-updater#xyz.iam.","client_id": "113522222222222222222222222","auth_uri": "https://accounts.google.com,"token_uri": "https://oauth.googleap,"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth/v1/certs","client_x509_cert_url": "https://www.googleapis.com/v1"}
username as _json_key and password is "json file content"
The error which I am getting is as below:-
error: error parsing argocd-image-updater-secret.yaml: error converting YAML to JSON: yaml: line 7: mapping values are not allowed in this context
You're getting bitten by a yaml-ism, as yaml2json or yamllint would inform you
Error: Cannot parse as YAML (mapping values are not allowed here
in "<byte string>", line 5, column 28:
creds: _json_key:{"type": "service_account","project_id" ...
^)
what you'll want is to fold that scalar so the : is clearly character data and not parsed as a yaml key
metadata:
name: image-secret
type: Opaque
stringData:
creds: >-
_json_key:{"type": "service_account","project_id": "xyz","private_key_id": "9b0eb25b41ae9161123dbfh56mgj","private_key": "-----BEGIN PRIVATE KEY-----\nmch0iiFz1DAdM8vQTXiETI+3gvSnknXQ0M5WmkA1dkiJgyhe3r8tpeb42jo4FCd\nbHLf9eeIql8TKEm9BAk+qnQZq8FykWEnQLuU7APrFNZ0qtYP8t1Y7HSGpdVmmCyK\nykJAGznKaiEf9SJiNy8HqJy1kOhajn1fL3CdcShWcY793qRLyeFyrIZ\n6lfnjSE9IW5iEOBmxEpXf5Q=\n-----END PRIVATE KEY-----\n","client_email": "argocd-image-updater#xyz.iam.","client_id": "113522222222222222222222222","auth_uri": "https://accounts.google.com,"token_uri": "https://oauth.googleap,"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth/v1/certs","client_x509_cert_url": "https://www.googleapis.com/v1"}
I need to create, among other items, an SQL instance on the google cloud platform. I am required to create the database using Jinja files.
My Yaml file
imports:
- path: companyInstance.jinja
resources:
# Creates a database instance
- name: dbinstance11
type: companyInstance.jinja
My companyInstance.jinja
resources:
- name: test
type: sqladmin.v1beta4.instance
properties:
zone: europe-west1
settings:
- tier: db-custom-4-15360
Currently, this should all work as far as I can tell but on a full deployment, I get an error message claiming that type is given an array type instead of object.
The full error is below:
ERROR: (gcloud.deployment-manager.deployments.create) Error in Operation [operation-169876402531-756fgfg342844d-6gd6ebb6-8bb655a]: errors:
- code: CONDITION_NOT_MET
location: /deployments/dep/resources/test->$.properties
message: |
error: instance type (array) does not match any allowed primitive type (allowed: ["object"]) level: "error"
schema: {"loadingURI":"#","pointer":"/schemas/Settings"}
instance: {"pointer":"/settings"}
domain: "validation"
keyword: "type"
found: "array"
expected: ["object"]
I'm writing AWS CloudFormation template (using yaml) which creates AWS Service Catalog Product.
I'm getting the template for the product using parameter S3FilePath which has a value like the above path: https://bucket.s3-eu-west-1.amazonaws.com/template.yml.
The URL to the file needs to be send in a JSON format as shown here (this example works):
Resources:
Type: AWS::ServiceCatalog::CloudFormationProduct
Properties:
Description: Example Product
Distributor: xyz
Name: ExampleProduct
Owner: xyz
ProvisioningArtifactParameters:
- Description: Example Product
Info: { "LoadTemplateFromURL": "https://bucket.s3-eu-west-1.amazonaws.com/template.yml" }
Name: Version1
I tried to replace the URL using !Sub and !Ref as shown below:
Parameters:
S3FilePath:
Type: String
Description: file name
Resources:
Type: AWS::ServiceCatalog::CloudFormationProduct
Properties:
Description: Example Product
Distributor: xyz
Name: ExampleProduct
Owner: xyz
ProvisioningArtifactParameters:
- Description: Example Product
Info: !Sub
- '{ "LoadTemplateFromURL": "${FILEPATH}" }'
- {FILEPATH: !Ref S3FilePath}
Name: Version1
But the CloudFormation stack fails with the error: "invalid input".
I guess I am building the JSON in a wrong way, I tried to use \ before each ' " ' but it didn't help either and I couldn't find an example which explain how to build this correctly. There is no problem with the S3FilePath parameter.
Can you please advice how to use the !Sub and !Ref correctly to build the JSON? Thanks.
Here is an example: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-sub.html#w2ab1c25c28c59c11
Despite the documentation saying the Info parameter is JSON, the example shows just a name/value pair (Map): https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-servicecatalog-cloudformationproduct-provisioningartifactproperties.html
Try formatting your string as
Info: !Sub
- "LoadTemplateFromURL": "${FILEPATH}"
- {FILEPATH: !Ref S3FilePath}
You can reference any Parameters or LogicalResourceId directly inside a !Sub like so:
ProvisioningArtifactParameters:
- Description: Example Product
Info: !Sub '{ "LoadTemplateFromURL": "${S3FilePath}" }'
Name: Version1
This should work totally fine. The way you were doing substitutions is useful when you want to use conditions and/or mapping inside a !Sub.
I think it should be simply:
ProvisioningArtifactParameters:
- Description: Example Product
Info:
LoadTemplateFromURL: !Ref S3FilePath
Name: Version1
This is at least what I have in my own AWS::ServiceCatalog::CloudFormationProduct templates.
ProvisioningArtifactParameters:
- DisableTemplateValidation: false
Info:
LoadTemplateFromURL: !Ref S3FilePath
I've a kubernetes cronjob manifest file.In that file I've defined enviornment variables.I'm generating yaml using a shell script but while using the yaml using kubectl create -f. I'm getting the following validation error
error validating "cron.yaml": error validating data: [ValidationError(CronJob.spec.jobTemplate.spec.template.spec.containers[0].envFrom[0].configMapRef): invalid type for io.k8s.api.core.v1.ConfigMapEnvSource: got "array", expected "map".
Can anyone suggest me how to resolve this?
You have a mistake in the syntax.
There are two approaches, using valueFrom for individual values or envFrom for multiple values.
valueFrom is used inside the env attribute.valueFrom will inject the value of a a key from the referenced configMap.
spec:
template:
spec:
containers:
- name: ad-sync
image: foo.azurecr.io/foobar/ad-sync
command: ["dotnet", "AdSyncService.dll"]
args: []
env:
- name: AdSyncService
valueFrom:
configMapKeyRef:
name: ad-sync-service-configmap
key: log_level
envFrom is used direct inside the container attribute.envFrom will inject All configMap keys as environment variables
spec:
template:
spec:
containers:
- name: ad-sync
image: foo.azurecr.io/foobar/ad-sync
command: ["dotnet", "AdSyncService.dll"]
envFrom:
- configMapRef:
name: ad-sync-service-configmap
Part of our orchestration uses envsubst to update a YAML template file with our desired values.
envsubst < "${SECRET_TEMPLATE}" | kubectl apply -f -
The value for our keyword config is a JSON string:
data=$(jq -c . ${JSON_FILE})
This results in YAML that looks like this (trimmed for brevity):
apiVersion: v1
kind: Secret
metadata:
name: reporting-config
type: Opaque
data:
config: {"database": "foo"}
This apparently worked in some earlier versions of Kube, I wanna say 1.8. Anyways, we are running 1.15 and now kubectl interprets this as a map type and complains:
error: error validating "STDIN": error validating data: ValidationError(Secret.data.config): invalid type for io.k8s.api.core.v1.Secret.data: got "map", expected "string"; if you choose to ignore these errors, turn validation off with --validate=false
Is there a trick to doing this now. I've played around with quoting and various places, escaping quotes, and all that jazz and nada.
* update 1 *
Using stringData still results in the same error:
apiVersion: v1
kind: Secret
metadata:
name: monsoon-storage-reporting-config
type: Opaque
stringData:
config: {"database": "foo"}
error: error validating "STDIN": error validating data: ValidationError(Secret.stringData.config): invalid type for io.k8s.api.core.v1.Secret.stringData: got "map", expected "string"; if you choose to ignore these errors, turn validation off with --validate=false
You may use stringData as below. Note the pipe after stringData:
apiVersion: v1
kind: Secret
metadata:
name: monsoon-storage-reporting-config
type: Opaque
stringData: |
config: {"database": "foo"}
I had to base64 encode the value
$ echo {"database": "foo"} | base64
e2RhdGFiYXNlOiBmb299Cg==
and then use the base64 encoded value in the data: field
apiVersion: v1
kind: Secret
metadata:
name: reporting-config
type: Opaque
data:
config: e2RhdGFiYXNlOiBmb299Cg==
Also note this on base64 encoding:
When using the base64 utility on Darwin/macOS users should avoid using the -b option to split long lines. Conversely Linux users should add the option -w 0 to base64 commands or the pipeline base64 | tr -d '\n' if -w option is not available.