RabbitMQ Shovel config with Alternate-Exchange - configuration

I'm trying to configure the Shovel plugin for RabbitMQ with a list of declarations. I have configured my remote exchange to have an alternate-exchange when I created it.
My problem is that I can't get the config file for shovel to include this argument so RabbitMQ crashes upon startup.
This is what my config looks like:
[
{mnesia, [{dump_log_write_threshold, 100}]},
{rabbit, [{vm_memory_high_watermark, 0.4}]},
{rabbitmq_shovel,
[{shovels,
[{call_stats_shovel,
[{sources, [{broker, "amqp://guest:guest#localhost:5672/test"},
{declarations,
[{'queue.declare', [{queue, <<"incoming">>}, durable]},
{'exchange.declare',[{exchange, <<"my-exchange-topic">>},{type, <<"topic">>},durable]},
{'queue.bind',[{exchange, <<"my-exchange-topic">>},{queue, <<"incoming">>}]}
]}]},
{destinations, [{broker, "amqp://guest:guest#172.16.3.162:5672/blah"},
{declarations,
[
{'queue.declare',[{queue, <<"billing">>},durable]},
{'exchange.declare',[{exchange, <<"my-exchange-topic">>},{type, <<"topic">>},{alternate_exchange, <<"alt">>}, durable]},
{'queue.bind',[{exchange, <<"my-exchange-topic">>},{queue, <<"billing">>},{routing_key, <<"physical">>}]}
]}
]},
{queue, <<"incoming">>},
{ack_mode, no_ack},
{publish_properties, [{delivery_mode, 2}]},
{reconnect_delay, 5}
]}
]
}]
}
].
The problem is on the destination exchange called my-exchange-topic. If I take out the declarations section then the config file works.
This is the error:
=INFO REPORT==== 31-Jul-2012::12:15:25 ===
application: rabbitmq_shovel
exited: {{invalid_shovel_configuration,call_stats_shovel,
{invalid_parameter_value,destinations,
{unknown_fields,'exchange.declare',
[alternate_exchange]}}},
{rabbit_shovel,start,[normal,[]]}}
type: permanent
If I leave the alternate_exchange section out of the declaration I get this error in RabbitMQ web management:
{{shutdown,
{server_initiated_close,406,
<<"PRECONDITION_FAILED - inequivalent arg 'alternate-exchange'for exchange 'my-exchange-topic' in vhost 'blah':
received none but current is the value 'alt' of type 'longstr'">>}},
{gen_server,call,
[<0.473.0>,
{call,
{'exchange.declare',0,<<"my-exchange-topic">>,<<"topic">>,false,
true,false,false,false,[]},
none,<0.444.0>},
infinity]}}

For anyone looking at how to configure exchanges and queues that require additional arguments you do it like this:
{'exchange.declare',[{exchange, <<"my-exchange-topic">>},{type, <<"topic">>}, durable, {arguments, [{<<"alternate-exchange">>, longstr, <<"alternate-exchange">>}]} ]},
you can do a similar thing with queues:
{'queue.declare',[{queue, <<"my-queue">>},durable, {arguments, [{<<"x-dead-letter-exchange">>, longstr, <<"dead-letter-queue">>}]}]}

For clarification of the comment above, in case of an exchange2exchange shovel, the config would be:
{'exchange.declare',[{exchange, <<"my-exchange-topic">>},{type, <<"topic">>}, durable, {arguments, [{<<"alternate-exchange">>, longstr, <<"name-of-your-alternate-exchange">>}]} ]},

Related

Importing JSON file for custom role in Azure AD

Trying to import a JSON file for creating an Azure custom role with the following content
{
"Name": "Veeam Backup for Microsoft Azure v4 Service Account Permissions",
"Id:": null
"IsCustom": true,
"Description": "Permissions needed to function Veeam Backup for Microsoft Azure v4",
"Actions": [
"Microsoft.Authorization/roleAssignments/read",
"Microsoft.Authorization/*/Write",
"Microsoft.Commerce/RateCard/read",
"Microsoft.Compute/disks/beginGetAccess/action",
"Microsoft.Compute/disks/delete",
"Microsoft.Compute/disks/endGetAccess/action",
"Microsoft.Compute/disks/read",
"Microsoft.Compute/disks/write",
"Microsoft.Compute/snapshots/beginGetAccess/action",
"Microsoft.Compute/snapshots/delete",
"Microsoft.Compute/snapshots/endGetAccess/action",
"Microsoft.Compute/snapshots/read",
"Microsoft.Compute/snapshots/write",
"Microsoft.Compute/virtualMachines/deallocate/action",
"Microsoft.Compute/virtualMachines/delete",
"Microsoft.Compute/virtualMachines/extensions/read",
"Microsoft.Compute/virtualMachines/extensions/write",
"Microsoft.Compute/virtualMachines/read",
"Microsoft.Compute/virtualMachines/runCommand/action",
"Microsoft.Compute/virtualMachines/start/action",
"Microsoft.Compute/virtualMachines/write",
"Microsoft.DevTestLab/Schedules/write",
"Microsoft.Network/networkInterfaces/delete",
"Microsoft.Network/networkInterfaces/join/action",
"Microsoft.Network/networkInterfaces/read",
"Microsoft.Network/networkInterfaces/write",
"Microsoft.Network/networkSecurityGroups/join/action",
"Microsoft.Network/networkSecurityGroups/read",
"Microsoft.Network/publicIPAddresses/join/action",
"Microsoft.Network/publicIPAddresses/read",
"Microsoft.Network/publicIPAddresses/delete",
"Microsoft.Network/publicIPAddresses/write",
"Microsoft.Network/virtualNetworks/read",
"Microsoft.Network/virtualNetworks/subnets/join/action",
"Microsoft.Network/virtualNetworks/write",
"Microsoft.Resources/subscriptions/resourceGroups/moveResources/action",
"Microsoft.Resources/subscriptions/resourceGroups/delete",
"Microsoft.Resources/subscriptions/resourceGroups/read",
"Microsoft.Resources/subscriptions/resourceGroups/write",
"Microsoft.ServiceBus/namespaces/queues/authorizationRules/ListKeys/action",
"Microsoft.ServiceBus/namespaces/queues/authorizationRules/read",
"Microsoft.ServiceBus/namespaces/queues/authorizationRules/write",
"Microsoft.ServiceBus/namespaces/queues/delete",
"Microsoft.ServiceBus/namespaces/queues/read",
"Microsoft.ServiceBus/namespaces/queues/write",
"Microsoft.ServiceBus/namespaces/read",
"Microsoft.ServiceBus/namespaces/write",
"Microsoft.ServiceBus/register/action",
"Microsoft.Sql/locations/*",
"Microsoft.Sql/managedInstances/databases/delete",
"Microsoft.Sql/managedInstances/databases/read",
"Microsoft.Sql/managedInstances/databases/write",
"Microsoft.Sql/managedInstances/encryptionProtector/read",
"Microsoft.Sql/managedInstances/read",
"Microsoft.Sql/servers/databases/azureAsyncOperation/read",
"Microsoft.Sql/servers/databases/read",
"Microsoft.Sql/servers/databases/transparentDataEncryption/read",
"Microsoft.Sql/servers/databases/usages/read",
"Microsoft.Sql/servers/databases/write",
"Microsoft.Sql/servers/databases/delete",
"Microsoft.Sql/servers/elasticPools/read",
"Microsoft.Sql/servers/read",
"Microsoft.Storage/storageAccounts/blobServices/read",
"Microsoft.Storage/storageAccounts/listKeys/action",
"Microsoft.Storage/storageAccounts/managementPolicies/write",
"Microsoft.Storage/storageAccounts/read",
"Microsoft.Storage/storageAccounts/write",
"Microsoft.Authorization/roleDefinitions/write",
"Microsoft.Sql/servers/encryptionProtector/read",
"Microsoft.Compute/diskEncryptionSets/read",
"Microsoft.KeyVault/vaults/read",
"Microsoft.KeyVault/vaults/keys/versions/read",
"Microsoft.KeyVault/vaults/deploy/action",
"Microsoft.Sql/servers/databases/syncGroups/read"
],
"NotActions": [],
"DataActions": [
"Microsoft.KeyVault/vaults/keys/read",
"Microsoft.KeyVault/vaults/keys/encrypt/action",
"Microsoft.KeyVault/vaults/keys/decrypt/action"
],
"NotDataActions": [],
"AssignableScopes": [
"/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx"
]
}
results in the following error message:
Operation returned an invalid status code 'Unauthorized'
For import I used this command:
New-AzRoleDefinition -InputFile "/home/christian/VBAZv4_CustomRole.json"
I'm quite sure that I have sufficient permission to import.
Any idea, why I'm not able to import the JSON file.
To Create and assign a custom role in Azure Active Directory you require the following:
Azure AD Premium P1 or P2 license
Privileged Role Administrator or Global Administrator
AzureADPreview module when using PowerShell
Admin consent when using Graph explorer for Microsoft Graph API
Source: https://learn.microsoft.com/en-us/azure/active-directory/roles/custom-create#prerequisites
Hope this helps!

Invalid JSON response from cPanel API

I am using the cPanel UAPI but for some reason I am not receiving a valid JSON response with the additional curly braces and other quotations. I have tried different commands and on different servers and it's the same thing so I'm sure it's something simple I'm missing. Here is the doc I am following: https://api.docs.cpanel.net/openapi/cpanel/operation/query/
I've worked with other JSON APIs and I've never seen this before so is it cPanel specific and how would I parse this?:
---
apiversion: 3
func: query
module: Bandwidth
result:
data:
example.com:
"1572580800": '2978386611'
"1575176400": '3017940650'
"1577854800": '1272949062'
"1580533200": '1305954799'
"1583038800": '1072552621'
"1585713600": '1080945766'
"1588305600": '1188271342'
"1590984000": '1209022420'
"1593576000": 825711372
"1596254400": 771882286
"1598932800": '1126013108'
"1601524800": '1739977967'
"1604203200": '1225180686'
"1606798800": '1260268241'
"1609477200": '1503779339'
"1612155600": 985363356
"1614574800": 289737623
subdomain.example.com:
"1572580800": 4054128
"1575176400": 4187711
"1577854800": 83997477
"1580533200": 239566187
"1583038800": 413164665
"1585713600": 247957953
"1588305600": 239197568
"1590984000": 237783617
"1593576000": 304554763
"1596254400": 255965489
"1598932800": 251708858
"1601524800": 264273039
"1604203200": 138803673
"1606798800": 121939910
"1609477200": 68686277
"1612155600": 74725743
"1614574800": 19763737
UNKNOWN:
"1572580800": 13288313
"1575176400": 6326053
"1577854800": 17962542
"1580533200": 9835755
"1583038800": 22566655
"1598932800": 25137467
"1604203200": 31783967
"1606798800": 28088240
"1609477200": 22907243
"1612155600": 21413429
"1614574800": 2085610
subdomain1.example.com:
"1604203200": 2107896
"1606798800": 2516570
"1609477200": 3454393
"1612155600": 3216589
"1614574800": 795065
subdomain2.example.com:
"1580533200": 557880
"1583038800": 440351
"1585713600": 183146
"1588305600": 1047856
"1590984000": 415107
"1593576000": 342764
"1596254400": 5887618
"1598932800": 429651
errors: ~
messages: ~
metadata: {}
status: 1
warnings: ~ ```
cPanel got back to me and all I needed to do was to add --output=jsonpretty at the end.

Converting Packer 1.6 vsphere-iso configuration code from JSON to HCL2

With the release of Packer 1.6 came several depreciated fields in the vsphere-iso builder. From the looks of it, seems to be a format/type change because the fields actually still exists but just as properties it seems. An example of the changes are the following:
Working in Packer 1.5.6:
JSON
"disk_size": 123456,
"disk_thin_provisioned": true
"network": "VM Network",
"network_card": "vmxnet3"
Working in Packer 1.6.0:
JSON
"storage": [
{
"disk_size": 123456,
"disk_thin_provisioned": true
}
],
"network_adapters": [
{
"network": "VM Network",
"network_card": "vmxnet3"
}
]
The issue I have at the moment is I'm using Packer 1.6.0 and am trying to convert the above working JSON code to HCL2. I can't figure out the HCL2 syntax that supports the changes that were made in Packer 1.6.0.
I've tried the following:
network_adapters = {
network_card = "vmxnet3"
network = "VM Network"
}
Output:
An argument named "network_adapter" is not expected here.
network_adapters = (
network_card = "vmxnet3"
network = "VM Network"
)
Output:
Error: Unbalanced parentheses
on .\Packer\ConfigFileName.pkr.hcl line 19, in source "vsphere-iso"
"Test": 18: storage = ( 19: disk_thin_provisioned = true
Expected a closing parenthesis to terminate the expression.
network_adapters = [
network_card = "vmxnet3",
network = "VM Network"
]
Output:
Error: Missing item separator
on .\Packer\ConfigFileName.pkr.hcl line 19, in source "vsphere-iso"
"Test": 18: storage = [ 19: disk_thin_provisioned =
true,
Expected a comma to mark the beginning of the next item.
I've also tried several other permutations of different collection syntax together with no luck so far. Any suggestions or tips would greatly be appreciated
The correct syntax is the following:
network_adapters {
network_card = "vmxnet3",
network = "VM Network"
}
Note that it's not using an assignment operator = between network_adapters and {
Credit goes to SwampDragons over on the Packer forums for pointing this out.
If you're interested in knowing why: There was a change to how maps are treated in HCL2 back in May 2020 with the release of Packer 1.5.6
core/hcl2: Maps are now treated as settable arguments as opposed to blocks. For example tags = {} instead of tags {} [GH-9035]
Reference: https://github.com/hashicorp/packer/blob/master/CHANGELOG.md#156-may-1-2020

How to define config file variables?

I have a configuration file with:
{path, "/mnt/test/"}.
{name, "Joe"}.
The path and the name could be changed by a user. As I know, there is a way to save those variables in a module by usage of file:consult/1 in
-define(VARIABLE, <parsing of the config file>).
Are there any better ways to read a config file when the module begins to work without making a parsing function in -define? (As I know, according to Erlang developers, it's not the best way to make a complicated functions in -define)
If you need to store config only when you start the application - you may use application config file which is defined in 'rebar.config'
{profiles, [
{local,
[{relx, [
{dev_mode, false},
{include_erts, true},
{include_src, false},
{vm_args, "config/local/vm.args"}]
{sys_config, "config/local/yourapplication.config"}]
}]
}
]}.
more info about this here: rebar3 configuration
next step to create yourapplication.config - store it in your application folder /app/config/local/yourapplication.config
this configuration should have structure like this example
[
{
yourapplicationname, [
{path, "/mnt/test/"},
{name, "Joe"}
]
}
].
so when your application is started
you can get the whole config data with
{ok, "/mnt/test/"} = application:get_env(yourapplicationname, path)
{ok, "Joe"} = application:get_env(yourapplicationname, name)
and now you may -define this variables like:
-define(VARIABLE,
case application:get_env(yourapplicationname, path) of
{ok, Data} -> Data
_ -> undefined
end
).

Postgrex how to define json library

I'm just trying to use Postgrex without any kind of ecto setup, so just the example from the documentation readme.
Here is what my module looks like:
defmodule Receive do
def start(_type, _args) do
{:ok, pid} = Postgrex.start_link(
hostname: "localhost",
username: "john",
# password: "",
database: "property_actions",
extensions: [{Postgrex.Extensions.JSON}]
)
Postgrex.query!(
pid,
"INSERT INTO actions (search_terms) VALUES ($1)",
[
%{foo: 'bar'}
]
)
end
end
when I run the code I get
** (RuntimeError) type `json` can not be handled by the types module Postgrex.DefaultTypes, it must define a `:json` library in its options to support JSON types
Is there something I'm not setting up correctly? From what I've gathered in the documentation, I shouldn't even need to have that extensions line because json is handled by default.
On Postgrex <= 0.13, you need to define your own types:
Postgrex.Types.define(MyApp.PostgrexTypes, [], json: Poison)
and then when starting Postgrex:
Postgrex.start_link(types: MyApp.PostgrexTypes)
On Postgrex >= 0.14 (currently master), it was made easier:
config :postgrex, :json_library, Poison