To create a new config group, it is mandatory to provide a config group name, a tag and the cluster name to which it belongs. The tag as seen in this example is a name of the service. Two config groups with the same tag cannot be associated with the same host.
how to run the following json file with curl ?
in order to set this config group in ambari
POST /api/v1/clusters/c1/config_groups
[
{
"ConfigGroup": {
"cluster_name": "c1",
"group_name": "hdfs-nextgenslaves",
"tag": "HDFS",
"description": "HDFS configs for rack added on May 19, 2010",
"hosts": [
{
"host_name": "host1"
}
],
"desired_configs": [
{
"type": "core-site",
"tag": "nextgen1",
"properties": {
"key": "value"
}
}
]
}
}
]
reference - https://github.com/swagle/test/blob/master/docs/api/v1/config-groups.md
Is your question about how to send multiline json with curl? You can find different methods here.
I have an ARM template that deploys API's to an API Management instance
Here is an example of one API
{
"properties": {
"authenticationSettings": {
"subscriptionKeyRequired": false
},
"subscriptionKeyParameterNames": {
"header": "Ocp-Apim-Subscription-Key",
"query": "subscription-key"
},
"apiRevision": "1",
"isCurrent": true,
"subscriptionRequired": true,
"displayName": "DDD.CRM.PostLeadRequest",
"serviceUrl": "https://test1/api/FuncCreateLead?code=XXXXXXXXXX",
"path": "CRMAPI/PostLeadRequest",
"protocols": [
"https"
]
},
"name": "[concat(variables('ApimServiceName'), '/mms-crm-postleadrequest')]",
"type": "Microsoft.ApiManagement/service/apis",
"apiVersion": "2019-01-01",
"dependsOn": []
}
When I am deploying this to different environments I would like to be able to substitute the service url depending on the environment. I'm wondering the best approach?
Can I read in a config file or something like that?
At the time of deployment I have a variable that tells me the environment so I can base decisions on that. Just not sure the best way to do it
See about ARM template parameters: https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-authoring-templates#parameters They can be specified in a separate file. So you will have single template, but environment specific parameter files.
I need to initialize from ~/myConfig.json, which looks like:
{
"databaseActive": "production",
"databases": [
{
"name": "localhost",
"PGDB": "asdf",
"PGHOST": "localhost",
"PGPASSWORD": "asdf",
"PGPORT": "5432",
"PGUSER": "asdf"
},
{
"name": "production",
"PGDB": "asdf",
"PGHOST": "asdf.rds.amazonaws.com",
"PGPASSWORD": "asdf",
"PGPORT": "5432",
"PGUSER": "asdf"
}
]
}
This means I cannot call scalikejdbc.config.DBs.setupAll(). How might I use this JSON file to initialize scalikeJDBC from the appropriate database settings, according to the value of databaseActive?
ScalikeJDBC offers only the HOCON reader. If you go with your own JSON config files, you need to write your own JSON parser which checks the databaseActive.
Parsing your config and binding it to ScalikeJDBC's config classess would be simple:
https://github.com/scalikejdbc/scalikejdbc/blob/3.3.5/scalikejdbc-config/src/main/scala/scalikejdbc/config/DBs.scala#L10-L17
https://github.com/scalikejdbc/scalikejdbc/blob/3.3.5/scalikejdbc-config/src/main/scala/scalikejdbc/config/TypesafeConfigReader.scala#L59-L124
I am working on an old web service where I generate the rest endpoints documentation that comply with OAS standards using a custom tool. Using this OAS json file I can deploy the API to Azure API Managements services through the portal and it all works fine. However, I need to automate this process and hence need to use ARM templates to deploy all web services to Azure APIM. I have been looking into the examples provided https://learn.microsoft.com/en-us/azure/templates/microsoft.apimanagement/service/apis but just can't seem to wrap my head around how to use a local OAS.json file or a file in github.
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Location for all resources."
}
}
},
"variables": {
"apiManagementServiceName": "price-capture"
},
"resources": [
{
"apiVersion": "2018-01-01",
"type": "Microsoft.ApiManagement/service/apis",
"name": "[variables('apiManagementServiceName')]",
"properties": {
"displayName": "Service display Name",
"apiRevision": "1",
"description": "API description",
//need help since it's not a swagger url
//wondering if there is a way to ref a local file like the option
//provided in the portal when we register api's manually.
"serviceUrl": "----",
"path": "----",
"protocols": [
"https"
],
"isCurrent": true,
"apiVersion": "v1",
"apiVersionDescription": "apiVersionDescription"
}
}
]
}
You can deploy and configure an entire API on API Management via ARM templates, but you cannot use a local file to provide the OpenApi/Swagger.
In your case the OpenApi/Swagger needs to be publicly accessible so the resource manager can read from it, so if the Github URL is freely accessible it should work.
I typically store the OpenApi/Swagger to a storage account and use the SAS token to access it from the ARM template.
You can check out this blog for details on automating API deployment in APIM:
https://blog.eldert.net/api-management-ci-cd-using-arm-templates-linked-template/
You can deploy the API using an Azure Resource Manager template of type Microsoft.ApiManagement/service/apis, and to use an Open API / swagger definition you need to specify the contentValue and and contentFormat parameters of the template
{
"name": "awesome-api-management/petstore",
"type": "Microsoft.ApiManagement/service/apis",
"apiVersion": "2018-06-01-preview",
"properties": {
"path": "petstore"
"contentValue": "petstore swagger file contents here", // or it's URL
"contentFormat": "swagger-json", // or swagger-link-json if externally available
}
}
I don't think it's possible to deploy the APIs configs via templates.
I've been trying to figure this out myself but I'm pretty sure you can't include the actual APIs you want in the service.
From what I can tell, you can't do that with the GIT repo either because that needs authentication that is manually created in the portal
I think the only thing you can automate with the ARM template is the actual API Management service and then you need to use the Azure API to add and configure the APIs on it.
However, I have yet to figure out how to do that myself.
I actually have a service ticket open to get help on that.
The API has changed slightly so this works:
The yaml file (calculatorApiFile) needs to be uploaded first to a blob storage, but this can be done as part of the deployment pipeline
{
"type": "Microsoft.ApiManagement/service/apis",
"apiVersion": "2019-01-01",
"name": "[concat(parameters('service_name'), '/b12b1d5ab8204cg6b695e3e861fdd709')]",
"dependsOn": [
"[resourceId('Microsoft.ApiManagement/service', parameters('service_name'))]"
],
"properties": {
"displayName": "Calculator",
"apiRevision": "1",
"description": "A simple Calculator ",
"path": "calc",
"value": "[concat(parameters('containerUri'), parameters('calculatorApiFile'), parameters('containerSasToken'))]",
"format": "openapi-link",
"protocols": [
"https"
],
"isCurrent": true
}
}
I figured out the answer ..all I had to do was write an azure function that fetches the oas.yaml file from a private github repository.
"variables":{
"swagger_json":"[concat(parameters('url_of_azurefunctionwithaccesskey'),'&&githuburi='parameter('raw_url'),'&githubaccesstoken=',parameter('personalaccesstoken')]"
},
"resources": [
{
"type": "Microsoft.ApiManagement/service/apis",
"name": "[concat(parameters('apimName') ,'/' ,parameters('serviceName'))]",
"apiVersion": "2018-06-01-preview",
"properties": {
"apiRevision": "[parameters('apiRevision')]",
"path": "pricecapture",
"contentValue": "[variables('swagger_json')]",
"contentFormat": "openapi-link"
}
}]
The Azure function that I had to write was something like this:
#r "Newtonsoft.Json"
using System.Net;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Primitives;
using Newtonsoft.Json;
using System.IO;
using System.Text;
public static async Task<HttpResponseMessage> Run(HttpRequest req, ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
var gitHubUri = req.Query["githuburi"];
var gitHubAccessToken = req.Query["githubaccesstoken"];
var encoding = Encoding.ASCII;
if (string.IsNullOrEmpty(gitHubUri))
{
var errorcontent = new StringContent("please pass the raw file content URI (raw.githubusercontent.com) in the request URI string", Encoding.ASCII);
return new HttpResponseMessage
{
StatusCode = HttpStatusCode.BadRequest,
Content = errorcontent
};
}
else if (string.IsNullOrEmpty(gitHubAccessToken))
{
var errorcontent = new StringContent("please pass the GitHub personal access token in the request URI string", Encoding.ASCII);
return new HttpResponseMessage
{
StatusCode = HttpStatusCode.BadRequest,
Content = errorcontent
};
}
else
{
var strAuthHeader = "token " + gitHubAccessToken;
var client = new HttpClient();
client.DefaultRequestHeaders.Add("Accept", "application/vnd.github.v3.raw");
client.DefaultRequestHeaders.Add("Authorization", strAuthHeader);
var response = await client.GetAsync(gitHubUri);
return response;
}
}
If you load your YAML into a variable, that can be passed to the ARM template and be passed as the value:
deploy.bat:
SETLOCAL EnableDelayedExpansion
set API_DEPLOYMENT=<deployment name>
set API_GROUP=<deployment group>
set API=<api file path.yml>
set OPENAPI=
for /f "delims=" %%x in ('type %API%') do set "OPENAPI=!OPENAPI!%%x\n"
call az deployment group create -n %API_DEPLOYMENT% -g %API_GROUP% --mode Complete -f deploy.json -p openApi="!OPENAPI!"
ENDLOCAL
deploy.json (note the use of replace)
...
{
"type": "Microsoft.ApiManagement/service/apis",
"apiVersion": "2020-12-01",
"name": "[variables('apiName')]",
"properties": {
"path": "[variables('service')]",
"apiType": "http",
"displayName": "[variables('apiDisplayName')]",
"format": "openapi",
"value": "[replace(parameters('openApi'), '\\n', '\n')]"
},
...
},
...
I have a packer json like:
"builders": [{...}],
"provisioners": [
{
"type": "file",
"source": "packer/myfile.json",
"destination": "/tmp/myfile.json"
}
],
"variables": {
"myvariablename": "value"
}
and myfile.json is:
{
"var" : "{{ user `myvariablename`}}"
}
The variable into the file does get replaced, is a sed replacement with shell provisioner after the file the only option available here?
Using packer version 0.12.0
You have to pass these as environment variables. For example:
"provisioners": [
{
"type": "shell"
"environment_vars": [
"http_proxy={{user `proxy`}}",
],
"scripts": [
"some_script.sh"
],
}
],
"variables": {
"proxy": null
}
And in the script you can use $http_proxy
So far I've come just with the solution to use file & shell provisioner. Upload file and then replace variables in file via shell provisioner which can be fed from template variables provided by e.g. HashiCorp Vault
Yo may use OS export function to set environment and pass it to Packer
Here is a config using OS ENV_NAME value to choose local folder to copy from
export ENV_NAME=dev will set local folder to dev
{
"variables": {
...
"env_folder": "{{env `ENV_NAME`}}",
},
"builders": [{...}]
"provisioners": [
{
"type": "file",
"source": "files/{{user `env_folder`}}/",
"destination": "/tmp/"
},
{...}
]
}
User variables must first be defined in a variables section within your template. Even if you want a user variable to default to an empty string, it must be defined. This explicitness helps reduce the time it takes for newcomers to understand what can be modified using variables in your template.
The variables section is a key/value mapping of the user variable name to a default value. A default value can be the empty string. An example is shown below:
{
"variables": {
"aws_access_key": "",
"aws_secret_key": ""
},
"builders": [{
"type": "amazon-ebs",
"access_key": "{{user `aws_access_key`}}",
"secret_key": "{{user `aws_secret_key`}}",
// ...
}]
}
check this link for more information