Kafka : Update jaas config dynamically - configuration

I have setup the jaas config for kafka using sasl.jaas.config property. I want to update this config and add users dynamically.
As per this doc - http://kafka.apache.org/11/documentation.html#dynamicbrokerconfigs, we can do that by using bin/kafka-configs.sh.
The above doc has config column, which says as follow -
I have tried updating sasl.jaas.config with below command:
bin/kafka-configs.sh --bootstrap-server localhost:9092 --entity-type brokers --entity-name 59 --alter --add-config sasl.jaas.config="KafkaServer {\n org.apache.kafka.common.security.plain.PlainLoginModule required\n username=\"myuser\"\n password=\"mypassword\";\n};\nClient {\n org.apache.zookeeper.server.auth.DigestLoginModule required\n username=\"myuser2\"\n password=\"mypassword2\";\n};"
But it gives me following error:
requirement failed: Invalid entity config: all configs to be added must be in the format "key=val"
If I look to above column, it says the format for value of sasl.jaas.config property is (=)*. What does this means?
How the value for 'sasl.jaas.config' should be passed to update jaas config dynamically?

While it's possible to dynamically update sasl.jaas.config to add more users, the default Plain login module is not intended to be used in production.
Instead you should define callback handlers to handle authentication of users. This is described in the Kafka Sasl Plain docs.
Another option that requires more work (but give even more flexibility) is to create your own login module. The process is described in Can Kafka be provided with custom LoginModule to support LDAP?
Regarding the error message you get, this seems to be an issue with the kafka-config.sh tool. It's not expecting the config value to contain =. You should be able to update that config using the AdminClient API.
I couldn't find an existing issue in JIRA so created a new one: https://issues.apache.org/jira/browse/KAFKA-8010

Related

ActiveMQ - How to set "jms.nonBlockingRedelivery" via connection url?

I am using Red Hat's AMQ 6 on OpenShift and consuming messages via Red Hat's EAP. I'm using the AMQ JCA resource adapter 5.11.1.
I want to set jms.nonBlockingRedelivery=true.
I can't find this in any of the following docs:
Connection Configuration URI
Resource Adapter Properties
Activation Spec Properties
Can this be set via a URL query parameter?
When I set a wrong value like:
<config-property name="ServerUrl">tcp://localhost:61616?...&jms.nonBlockingRedelivery=WWWWWWW1
The log says:
org.apache.activemq.ra.ActiveMQEndpointWorker: Successfully established connection to broker...&jms.nonBlockingRedelivery=WWWWWWW1]
I would have expected a complaint e.g. "no boolean," "can't parse," etc.
Setting nonBlockingRedelivery via URL query parameter is supported. You can do so using the jms. prefix, e.g. jms.nonBlockingRedelivery=true.
The reason you don't see any exception is because the OpenWire JMS client ultimately uses Boolean.valueOf(String) to evaluate WWWWWWW1 which simply returns false.
I added this to the documentation.

Why my Soffid JSON REST Web Services Connector does not update an object in the target system?

I am trying to connect my Soffid 3 server with our custom web application named Schrift. I am using а JSON REST Web Services Connector for this purpose. I added REST Web service plugin and then configured an agent with JSON/XML/SOAP Rest webservice type.
Loading of objects is working fine. My REST connector connects to the web service successfully and gets data of the accounts.
The problem is when I am trying to update some data (for example, I am trying to lock an account), nothing happens. And unfortunately I don't know what should be happening. When should REST connector send updated data to the managed system and in which way? I didn't find any log entries saying that REST connector was trying to update an object on managed system. Maybe I did smth wrong or missed something.
I would appreciate for any help. I can post any conf or log details if you need.
Update#1
(I did some investigation after the first answer)
I checked the agent settings: Read only and Manual account creation are set to no
The account was set to unmanaged type, but I succeeded in changing its type to shared and then to single without getting an error. Now it is set to single
The task queue is empty.
Also I've checked that update method is present and update properties are set correctly. updateParams is not set (it means that all attributes should be sent to the managed system).
But when I change status of the account (from Enable to Disable), nothing happens.
In the console log I can see only these lines
14-Sep-2021 13:26:29.708 INFO [BPM-Scheduler:192.168.7.121:1] com.soffid.iam.bpm.job.JobExecutorThread.run No job to execute
When I manually run the task Analize impact for changes on Schrift, Execution log shows
Changes detected for accounts
=============================
NO CHANGE DETECTED
Changes detected for roles
=============================
NO CHANGE DETECTED
Update#2
After many attempts I made some progress. Now when I make some changes in the account, the task named UpdateAccount baklykov#irf.com.ua#Schrift appears, but runs with an error.
At first it was 415 Unsupported Media Type error as I wrote in comments, but now it looks a little different
Throws exception updating object : Extensible object [type = account]
EmployeeEmail: baklykov#irf.com.ua
IsLockedOut: true (log truncated) ...
caused by Unexpected response, Content-Type: null
Update#3
I found out that soffid's request for updating the object was in improper format (all the parameters were passed in the html request instead of putting them in json body)
After researching I found a method's property called Encoding and set it to application/json value.
Now the parameters are passed in json body (that's what I need), but now the problem is that soffid puts all the parameters in json body, including the key parameter by which the object for updating should be determined. My guess this is the reason why the object in the target system is still not updated.
In other words my application expects a request like this:
https://myapp.mysite.com/api/v1/Soffid/Employees?EmployeeEmail=baklykov%40irf.com.ua :
{"EmployeeLastName":"Baklykov","EmployeeFirstName":"Ivan"}
but Soffid sends this:
https://myapp.mysite.com/api/v1/Soffid/Employees:
{"EmployeeLastName":"Baklykov","EmployeeFirstName":"Ivan","EmployeeEmail":"baklykov#irf.com.ua"}
The system should have created a UpdateAccount task in the task queue. Please, verify:
The task engine is in automatic mode. In read-only or manual mode, no task will be created.
If you are updating an account, check the account is not set as unmanaged. In that case, no tasks is created.
Finally, verify the task queue has not held the task up.
Have you checked the engine mode? Look at Main Menu > Administration > Configure Soffid > Integration engine > Smart engine settings
It should be set to automatic.

Azure linux VM deployment .json failing with inputting a password for deployment

Apologies in advance I am extremely new to this I am trying to run the following code and don't understand the error when attempting to generate a password for his azuredeploy.json runbook (or whatever its called)
$ az group deployment create --resource-group ProjectSpaghetti --template-uri https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/docker-simple-on-ubuntu/azuredeploy.jsonubuntu/azuredeploy.json
This command is implicitly deprecated because command group 'group deployment' is deprecated and will be removed in a future release. Use 'deployment group' instead.
Please provide string value for 'adminUsername' (? for help): luke
Please provide string value for 'dnsNameForPublicIP' (? for help): lasagne
Please provide securestring value for 'adminPasswordOrKey' (? for help):
Deployment failed. Correlation ID: 55abf44d-92e4-4a7f-9c0e-ed7e5353d586. {
"error": {
"code": "InvalidParameter",
"message": "The value of parameter linuxConfiguration.ssh.publicKeys.keyData is invalid.",
"target": "linuxConfiguration.ssh.publicKeys.keyData"
}
}
I assumed because it used the phrase "adminPasswordOrKey" I would be able to use a password of reasonable complexity. Is that not the case? I'm a real newb at this just having a play really.
You see this error because you seemed to have provided a password instead of the default SSH Public key for the adminPasswordOrKey parameter that this template parameter expects.
You could take either of the following steps next:
Pass on the SSH Public key it expects
If you do not already have one, generate a new keypair and pass the contents of the public key as follows:
az group deployment create --resource-group <resource-group-name> --template-uri https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/docker-simple-on-ubuntu/azuredeploy.json --parameters adminPasswordOrKey="ssh-rsa AAAAB..."
Specify an alternate authentication mechanism
Override the authenticationType to accept a password instead of a sshPublicKey. Check this blog for a quick walk through.
You could also set your parameters using a parameter file. Check this document for other ways: Deploy resources with ARM templates and Azure CLI

Spinnaker Jenkins Integration unable to fetch jobs from Jenkins

We have completed all the steps as described in the hello-spinnaker example below.We have used the AWS spinnaker image to directly configure spinnaker in AWS.
www.spinnaker.io/docs/hello-spinnaker.
I am trying to create a sample pipeline as noted in the above example.But while I create trigger in the first step and select jenkins ,the jobs are not getting populated and am getting below error in browser.
GET http://localhost:8084/v2/builds/Jenkins/jobs 429 (Too Many Requests)
The actual issue looks like while retrofit is trying to map the response from jenkins getjobs into the JobList class its finding an attribute _class in jenkins response xml and which is not present in JobList groovy class.Below is how we tried finding the issue
1)Login to AWS Spinnaker instance
2)Gate service is exposed at port 8084.
curl http://localhost:8084/v2/builds/Jenkins/jobs.
{"failureCause":"retrofit.RetrofitError: 429 Too Many Requests","error":"Too Many Requests","message":"429 Too Many Requests","status":429,"url":"http://localhost:8088/jobs/Jenkins","timestamp":1462793944530}
3)Igor service is exposed at port 8088.
curl http://localhost:8088/jobs/Jenkins
{"fallbackException":"java.lang.UnsupportedOperationException: No fallback available.","failureType":"COMMAND_EXCEPTION","failureCause":"retrofit.converter.ConversionException: org.simpleframework.xml.core.AttributeException: Attribute '_class' does not have a match in class com.netflix.spinnaker.igor.jenkins.client.model.JobList at line 1","error":"Hystrix Failure","message":"jenkins-Jenkins-getJobs failed and no fallback available.","status":429,"timestamp":1462793896853}
When I check in the igor logs,there are few exceptions which are occuring during the getprojects by jenkins poll
Caused by: retrofit.converter.ConversionException: org.simpleframework.xml.core.AttributeException: Attribute '_class' does not have a match in class com.netflix.spinnaker.igor.jenkins.client.model.ProjectsList at line 2
at retrofit.converter.SimpleXMLConverter.fromBody(SimpleXMLConverter.java:38)
at retrofit.RestAdapter$RestHandler.invokeRequest(RestAdapter.java:367)
... 39 common frames omitted
Caused by: org.simpleframework.xml.core.AttributeException: Attribute '_class' does not have a match in class com.netflix.spinnaker.igor.jenkins.client.model.ProjectsList at line 2
4)Connect to jenkins and get the jobs as its being done in spinnaker code https://github.com/spinnaker/igor/blob/master/igor-web/src/main/groovy/com/netflix/spinnaker/igor/jenkins/client/JenkinsClient.groovy
resp = requests.get('http://jenkinserverip:8080/api/xml?tree=jobs[name,jobs[name,jobs[name,jobs[name,jobs[name,jobs[name,jobs[name,jobs[name,jobs[name,jobs[name]]]]]]]]]]',auth=('admin','password'))
print resp.text
<hudson _class='hudson.model.Hudson'><job _class='hudson.model.FreeStyleProject'><name>Hello Build</name></job><job _class='hudson.model.FreeStyleProject'><name>Hello Poll</name></job></hudson>
So as the jenkins response is having the _class attribute ,retrofit is throwing an error at this line http://grepcode.com/file/repo1.maven.org/maven2/com.squareup.retrofit/retrofit/1.9.0/retrofit/RestAdapter.java#383
I wanted to see how can we quickly fix this as it looks like some version in compatibility of jenkins.
I'm seeing a similar issue in spinnaker 1.8.5. I had to reformat the jenkins url from myjenkins.server.com:8080 to http://myjenkins.server.com/ and it corrected the issue.
this is a bug around the jenkins api in later version. I believe 2.2 is the last compatible version, we run 1.6 internally.

play activator - runtimeException: smtp.host needs to be set in application.conf in order to use this plugin (or set smtp.mock to true)

I am trying to run play activator template named "playStartApp".
But, it is giving me following runtime exception:
RuntimeException: smtp.host needs to be set in application.conf in order to use this plugin (or set smtp.mock to true)\
java.lang.RuntimeException: smtp.host needs to be set in application.conf in order to use this plugin (or set smtp.mock to true)
com.typesafe.plugin.CommonsMailerPlugin$$anonfun$4.apply(MailerPlugin.scala:329)
com.typesafe.plugin.CommonsMailerPlugin$$anonfun$4.apply(MailerPlugin.scala:329)
scala.Option.getOrElse(Option.scala:120)
com.typesafe.plugin.CommonsMailerPlugin.mailerInstance$lzycompute(MailerPlugin.scala:329)
com.typesafe.plugin.CommonsMailerPlugin.mailerInstance(MailerPlugin.scala:326)
com.typesafe.plugin.CommonsMailerPlugin.onStart(MailerPlugin.scala:343)
play.api.Play$$anonfun$start$1$$anonfun$apply$mcV$sp$1.apply(Play.scala:91)`
I tried both:
- set mail.smtp=mock
- mail.smtp.host=smtp.gmail.com
mail.smtp.user=yourGmailLogin
mail.smtp.pass=yourGmailPassword
mail.smtp.channel=ssl
pls suggest how to resolve this?
In the "playStartApp" activator template, you can either -
Use a mock mailer, by entering the following in conf/application.conf
smtp.mock=true
The mock option will render email output to the console. If you are using the Activator UI, you'll be able to see the mail output in the logs on the "Run" tab.
Mention the SMTP server details in conf/application.conf
smtp.host=smtp.gmail.com
smtp.port=587
smtp.user="user#gmail.com"
smtp.password="password"
smtp.ssl=true
mail.from="PlayStartApp#yourdomain.com"
mail.sign=The PlayStartApp Team
You can use gmail servers for sending mails as well, which I have mentioned above.
Alternatively, in the "playStartApp", just rename conf/mail.conf.example to conf/mail.conf and move all your SMTP related configuration here. Please note that conf/mail.conf is mentioned in .gitignore.
Resources (Play Documentation):
Mail configuration parameters
SMTP Configuratoin