I am trying to get data with rundeck webhook plugin, and for this i am usig curl command:
curl -X POST -d '{"name":"John", "age":30, "car":null}' https://rundeck_server/api/12/webhook/QSxTDYd08dcYxKh1R5YJNOPQvmSJH2Z8#Netbox_Job
In rundeck webhook plugin options i add those 2 variables, 'whkpayload' to get all the json data and 'name' to get the name only (must return John in this example):
-whkpayload ${raw} -name ${data.name}
And finally i show them with those lines:
echo #option.whkpayload#
echo #option.name#
I get an empty result and i can't figure out why. Any one may help me please ?
Following this, you need to use an option called whkpayload in your job, and set it as ${raw} in the webhook configuration.
I made an example:
The job definition in YAML format (with the whkpayload option):
- defaultTab: nodes
description: ''
executionEnabled: true
id: 0fcfca07-02f6-4583-a3eb-0002276bdf2d
loglevel: INFO
name: HelloWorld
nodeFilterEditable: false
options:
- name: age
- name: car
- name: name
- name: whkpayload
plugins:
ExecutionLifecycle: null
scheduleEnabled: true
sequence:
commands:
- description: command step
exec: echo "name ${option.name} - age ${option.age} - car ${option.car} - payload
${option.whkpayload}"
- description: inline-script step
fileExtension: .sh
interpreterArgsQuoted: false
script: echo "name #option.name# - age #option.age# - car #option.car# - payload
#option.whkpayload#"
scriptInterpreter: /bin/bash
keepgoing: false
strategy: node-first
uuid: 0fcfca07-02f6-4583-a3eb-0002276bdf2d
The webhook configuration.
The webhook calling from cURL:
curl -H "Content-Type: application/json" -X POST -d '{"field1":"John", "field2":30, "field3":"chevy"}' http://localhost:4440/api/40/webhook/0vBZjWWrnXWvqENEdxkn0JRvjn5R63J0#MyWebhook
The result.
I am using Ansible to test some APIs. Instead of using get_url or uri module of ansible, i am using Curl requests with command/shell module.
I would need to extract a particular value from the json response of a curl api call, so that i can use that value in next curl request.
For example,
- name: Run a curl request to login and get jwt token
command: curl -k -X POST -H "Content-Type:application/json" --data '{"user":"{{username}}","password":"{{password}}"}' https://localhost:8001/api/v1/login
register: login_response
- name: Print the jwt response
debug:
var: login_response.stdout
- name: Run a curl request to get service token
command: curl -k -X GET -H "Accept:application/json" -H "Authorization:Bearer {{login_response.stdout.jwt}}" https://localhost:8001/api/v1/servicetoken/issue
register: service_token
- name: Print service token
debug:
var: service_token.stdout
The above ansible playbook will fail at third task because it cannot find a variable like:
login_response.stdout.jwt
Normally the variable in second task, ie, login_response.stdout will print something like this:
ok: [localhost] => {
"login_response.stdout": {
"jwt": "eyJhbGciOiJIUzUxMiIsImtpZCI6ImFiNmU0ZDg2LWE4YzgtNDU4OS04MmRiLWIxZTg1YzQwNDNlZiJ9.eyJleHAiOjE2MDU2MTIxODAuNjA5MTI1NCwiaWF0IjoxNjA1NjExODgwLjYwOTEyMC42MDkxMjU0fQ.ZC4a3H3j03ZmzDkjGj11cvxSls2qXZmVOGuIvKp8LHVpYOUyEJlWJJOTArHxKhxne3DsuqWoGpslR6KxuUOBFg",
"roles": [
{
"name": "Admin"
}
]
}
}
What i want is to extract that jwt token from the above response and use it in third task.
How can i achieve this?
I found the answer!
This can be achieved using from_json filter in order to make ansible able to access the json fields.
Now the working playbook should be like:
- name: Run a curl request to login and get jwt token
command: curl -k -X POST -H "Content-Type:application/json" --data '{"user":"{{username}}","password":"{{password}}"}' https://localhost:8001/api/v1/login
register: login_response
- name: Print the jwt token
debug:
var: (login_response.stdout|from_json).jwt
- name: Run a curl request to get service token
command: curl -k -X GET -H "Accept:application/json" -H "Authorization:Bearer {{(login_response.stdout|from_json).jwt}}" https://localhost:8001/api/v1/servicetoken/issue
register: service_token
- name: Print service token
debug:
var: (service_token.stdout|from_json).token
I refered here to get the idea: https://medium.com/#justinhennessy/parsing-json-with-ansible-bcbb8d4b6a54
I'm setting up a ejabberd-19.02 server on windows 10, configuring it to call admin commands through rest-api. I have followed the documentation as in ejabberd-doc. I am also able to generate oauth token.
http://localhost:5443/oauth/authorization_token?access_token=0GXCPVJa0l3jFH3Yrn9dIZFMftrNTBFa&token_type=bearer&expires_in=31536000&scope=ejabberd:admin&state=
But on calling rest-api method, using curl command as in documentation:
curl -v -X POST -H "X-Admin: true" -H "Authorization: Bearer 0GXCPVJa0l3jFH3Yrn9dIZFMftrNTBFa" http://localhost:5443/api/get_roster -d '{"user": "anas", "server": "localhost"}'
I am getting message as:
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying ::1...
* TCP_NODELAY set
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 5443 (#0)
> POST /api/get_roster HTTP/1.1
> Host: localhost:5443
> User-Agent: curl/7.55.1
> Accept: */*
> X-Admin: true
> Authorization: Bearer 0GXCPVJa0l3jFH3Yrn9dIZFMftrNTBFa
> Content-Length: 7
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 7 out of 7 bytes
< HTTP/1.1 400 Bad Request
< Content-Length: 20
< Content-Type: application/json
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Headers: Content-Type, Authorization, X-Admin
<
"Invalid JSON input"* Connection #0 to host localhost left intact
Note: Unnecessary use of -X or --request, POST is already inferred.
* Rebuilt URL to: anas,/
* Could not resolve host: anas,
* Closing connection 1
curl: (6) Could not resolve host: anas,
Note: Unnecessary use of -X or --request, POST is already inferred.
* Rebuilt URL to: server:/
* Could not resolve host: server
* Closing connection 2
curl: (6) Could not resolve host: server
curl: (3) [globbing] unmatched close brace/bracket in column 10
ejabberd.log says:
2019-08-07 13:52:08.201 [debug] <0.841.0>#ejabberd_http:init:151 S: [{[<<"api">>],mod_http_api},{[<<"bosh">>],mod_bosh},{[<<"upload">>],mod_http_upload},{[<<"ws">>],ejabberd_http_ws},{[<<"oauth">>],ejabberd_oauth},{[<<"admin">>],ejabberd_web_admin}]
2019-08-07 13:52:08.202 [info] <0.695.0>#ejabberd_listener:accept:238 (<0.841.0>) Accepted connection 127.0.0.1:59208 -> 127.0.0.1:5443
2019-08-07 13:52:08.203 [debug] <0.841.0>#ejabberd_http:process_header:307 (#Port<0.54>) http query: 'POST' <<"/api/get_roster">>
2019-08-07 13:52:08.203 [debug] <0.841.0>#ejabberd_http:process:394 [<<"api">>,<<"get_roster">>] matches [<<"api">>]
2019-08-07 13:52:08.203 [debug] <0.841.0>#mod_http_api:process:198 Bad Request: {error,{1,invalid_json}}
I have tried ejabberd version 19.06 and 19.08.
ejabberd.yml
language: "en"
hosts:
- "localhost"
loglevel: 4
log_rotate_size: 10485760
log_rotate_date: ""
log_rotate_count: 1
log_rate_limit: 100
sql_type: mysql
sql_server: "localhost"
sql_database: "ejabberd"
sql_username: "ejabberd"
sql_password: "#######"
## If you want to specify the port:
sql_port: 3306
default_db: sql
auth_method: sql
certfiles:
- "C:/ProgramData/ejabberd/conf/server.pem"
## - "/etc/letsencrypt/live/localhost/fullchain.pem"
## - "/etc/letsencrypt/live/localhost/privkey.pem"
ca_file: "C:/ProgramData/ejabberd/conf/cacert.pem"
listen:
-
port: 5222
module: ejabberd_c2s
max_stanza_size: 262144
shaper: c2s_shaper
access: c2s
starttls_required: true
-
port: 5269
module: ejabberd_s2s_in
max_stanza_size: 524288
-
port: 5443
module: ejabberd_http
request_handlers:
"/api": mod_http_api
"/bosh": mod_bosh
"/upload": mod_http_upload
"/ws": ejabberd_http_ws
"/oauth": ejabberd_oauth
web_admin: true
captcha: false
tls: false
-
port: 5280
module: ejabberd_http
web_admin: true
-
port: 1883
module: mod_mqtt
backlog: 1000
s2s_use_starttls: optional
acl:
local:
user_regexp: ""
loopback:
ip:
- "127.0.0.0/8"
admin:
user:
- "admin#localhost"
access_rules:
local:
- allow: local
c2s:
- deny: blocked
- allow
announce:
- allow: admin
configure:
- allow: admin
muc_create:
- allow: local
pubsub_createnode:
- allow: local
register:
- allow
trusted_network:
- allow: loopback
api_permissions:
"console commands":
from:
- ejabberd_ctl
who: all
what: "*"
"admin access":
who:
- access:
- allow:
- acl: loopback
- acl: admin
- oauth:
- scope: "ejabberd:admin"
- access:
- allow:
- acl: loopback
- acl: admin
what:
- "*"
- "!stop"
- "!start"
"public commands":
who:
- ip: "127.0.0.1/8"
what:
- "status"
- "connected_users_number"
commands_admin_access:
- allow:
- user: "admin#localhost"
commands:
- add_commands: [user, admin, open]
oauth_expire: 31536000
oauth_access:
- allow:
- user: "admin#localhost"
shaper:
normal: 1000
fast: 50000
shaper_rules:
max_user_sessions: 10
max_user_offline_messages:
- 5000: admin
- 100
c2s_shaper:
- none: admin
- normal
s2s_shaper: fast
max_fsm_queue: 10000
acme:
contact: "mailto:admin#localhost"
ca_url: "https://acme-v01.api.letsencrypt.org"
modules:
mod_adhoc: {}
mod_admin_extra: {}
mod_announce:
access: announce
mod_avatar: {}
mod_blocking: {}
mod_bosh: {}
mod_caps: {}
mod_carboncopy: {}
mod_client_state: {}
mod_configure: {}
mod_disco: {}
mod_fail2ban: {}
mod_http_api: {}
mod_http_upload:
put_url: "https://#HOST#:5443/upload"
mod_last: {}
## mod_mam:
## Mnesia is limited to 2GB, better to use an SQL backend
## For small servers SQLite is a good fit and is very easy
## to configure. Uncomment this when you have SQL configured:
## db_type: sql
## assume_mam_usage: true
## default: always
mod_mqtt: {}
mod_muc:
access:
- allow
access_admin:
- allow: admin
access_create: muc_create
access_persistent: muc_create
default_room_options:
allow_subscription: true # enable MucSub
## mam: true
mod_muc_admin: {}
mod_offline:
access_max_user_messages: max_user_offline_messages
mod_ping: {}
mod_privacy: {}
mod_private: {}
mod_proxy65:
access: local
max_connections: 5
mod_pubsub:
access_createnode: pubsub_createnode
plugins:
- "flat"
- "pep"
force_node_config:
## Change from "whitelist" to "open" to enable OMEMO support
## See https://github.com/processone/ejabberd/issues/2425
"eu.siacs.conversations.axolotl.*":
access_model: whitelist
## Avoid buggy clients to make their bookmarks public
"storage:bookmarks":
access_model: whitelist
mod_push: {}
mod_push_keepalive: {}
mod_register:
## Only accept registration requests from the "trusted"
## network (see access_rules section above).
## Think twice before enabling registration from any
## address. See the Jabber SPAM Manifesto for details:
## https://github.com/ge0rg/jabber-spam-fighting-manifesto
ip_access: trusted_network
mod_roster:
versioning: true
mod_s2s_dialback: {}
mod_shared_roster: {}
mod_stream_mgmt:
resend_on_timeout: if_offline
mod_vcard: {}
mod_vcard_xupdate: {}
mod_version:
show_os: false
You attempt to provide JSON data, but as you can see, curl sends only 7 bytes of content. 7 bytes are only 7 characters, see this example:
$ curl -v -X POST -H "X-Admin: true" -H "Authorization: Bearer UsGV4Ap1zHaaNxJNLiYncn9QG5TRXfEW" http://localhost:5443/api/get_roster -d '{"abcde'
.....
> Content-Length: 7
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 7 out of 7 bytes
....
So there must be some problem in the console or program you use to run curl, which cuts the command line. In this example, as you can see, a good JSON data takes 40 bytes (in my case the problem is that I didn't set admin rights):
$ curl -v -X POST -H "X-Admin: true" -H "Authorization: Bearer UsGV4Ap1zHaaNxJNLiYncn9QG5TRXfEW" http://localhost:5443/api/get_roster -d '{"user": "user1", "server": "localhost"}'
Note: Unnecessary use of -X or --request, POST is already inferred.
* Expire in 1 ms for 1 (transfer 0x557900f65a30)
* Trying 127.0.0.1...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x557900f65a30)
* Connected to localhost (127.0.0.1) port 5443 (#0)
> POST /api/get_roster HTTP/1.1
> Host: localhost:5443
> User-Agent: curl/7.64.0
> Accept: */*
> X-Admin: true
> Authorization: Bearer UsGV4Ap1zHaaNxJNLiYncn9QG5TRXfEW
> Content-Length: 40
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 40 out of 40 bytes
< HTTP/1.1 403 Forbidden
< Content-Length: 111
< Content-Type: application/json
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Headers: Content-Type, Authorization, X-Admin
<
* Connection #0 to host localhost left intact
{"status":"error","code":32,"message":"AccessRules: Account does not have the right to perform the operation."}
When I use the following command for the ejabberd API I get the following response;
curl -ik -X POST -H 'Authorization: Bearer xxxxxxxxxxx' https://localhost:5280/api/set_vcard -d '{"user":"foo","host":"example.com","name":"FN","content":"foobar"}'
HTTP/1.1 400 Bad Request
Content-Length: 18
Content-Type: application/json
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Content-Type, Authorization, X-Admin
"database_failure"
On the ejabberd log (level 5) I see this;
[info] (<0.607.0>) Accepted connection ::ffff:172.18.0.1:46622 -> ::ffff:172.18.0.3:5280
[debug] S: [{[<<"ws">>],ejabberd_http_ws},{[<<"bosh">>],mod_bosh},{[<<"oauth">>],ejabberd_oauth},{[<<"api">>],mod_http_api},{[<<"admin">>],ejabberd_web_admin}]
[debug] ({tlssock,#Port<0.18819>,#Ref<0.650175335.3240493057.203147>}) http query: 'POST' <<"/api/set_vcard">>
[debug] client data: <<"{\"user\":\"foo\",\"host\":\"example.com\",\"contents\":[\"FN:foobar\"]}">>
[debug] [<<"api">>,<<"set_vcard">>] matches [<<"api">>]
[info] API call set_vcard [{<<"user">>,<<"foo">>},{<<"host">>,<<"example.com">>},{<<"contents">>,[<<"FN:foobar">>]}] from ::ffff:172.18.0.1:46622
[debug] Command 'set_vcard' execution allowed by rule 'api service' (CallerInfo=#{caller_module => mod_http_api,caller_server => <<"example.com">>,ip => {0,0,0,0,0,65535,44050,1},oauth_scope => [<<"ejabberd:api-service">>],usr => {<<"admin">>,<<"example.com">>,<<>>}})
[debug] Executing command mod_admin_extra:set_vcard with Args=[<<"foo">>,<<"example.com">>,<<>>,<<>>,[<<"FN:foobar">>]]
It is using MySQL as a database (working fine for everything else) however when I watch the database general query log I don't see my API request trigger any queries. I see all the other normal ejabberd queries so there isn't a problem with the db connection and as mentioned earlier everything else works.
$ ejabberdctl status
The node ejabberd#e87da11aa894 is started with status: started
ejabberd 18.4.0 is running in that node
Does anyone have any clues they can throw my way? I've ran out of leads on what could be the issue.
!!! EDIT !!!
Work around
As mentioned in https://github.com/processone/ejabberd/issues/2629 other people have experienced this issue. Changing config to disable the cache and clearing the vcard table in the database seems to be a work around;
SQL:
DELETE FROM vcard;
Config:
...
mod_vcard:
search: false
use_cache: false
...
The API is fairly permissive in what it allows, however once it's in the database the record will fail to load.
For 'set_vcard', the 'name 'is the field name you wish to alter and the content is the contents of that field.
{
"user": "catman",
"host": "the.host",
"name": "FN",
"content": "Cat Man"
}
ejabberd also caches queries, so once you have a barfed record it'll return 'database_failed' even if you've corrected your api call or fixed it in the database by hand. Caching can be disabled under the modules configuration.
Notice in your log that it says:
[debug] client data: <<"{\"user\":\"foo\",\"host\":\"example.com\",\"contents\":[\"FN:foobar\"]}">>
How can it be that contents is FN:foobar? I installed 18.04, setup mysql storage, and running this query:
$ curl -v -H "X-Admin: true" -H "Content-Type:application/json" http://localhost:5280/api/set_vcard -d '{"user":"user1","host":"localhost","name":"FN","content":"mi nombre curllll"}'
The log says:
21:42:29.638 [info] (<0.487.0>) Accepted connection 127.0.0.1:58412 -> 127.0.0.1:5280
21:42:29.638 [debug] S: [{[<<"api">>],mod_http_api},{[<<"bosh">>],mod_bosh},{[<<"oauth">>],ejabberd_oauth},{[<<"presence">>],mod_webpresence},{[<<"register">>],mod_register_web},{[<<"rest">>],mod_rest},{[<<"ws">>],ejabberd_http_ws},{[<<"admin">>],ejabberd_web_admin}]
21:42:29.639 [debug] (#Port<0.18079>) http query: 'POST' <<"/api/set_vcard">>
21:42:29.639 [debug] client data: <<"{\"user\":\"user1\",\"host\":\"localhost\",\"name\":\"FN\",\"content\":\"mi nombre curllll\"}">>
21:42:29.639 [debug] [<<"api">>,<<"set_vcard">>] matches [<<"api">>]
21:42:29.639 [info] API call set_vcard [{<<"user">>,<<"user1">>},{<<"host">>,<<"localhost">>},{<<"name">>,<<"FN">>},{<<"content">>,<<"mi nombre curllll">>}] from 127.0.0.1:58412
21:42:29.640 [debug] Command 'set_vcard' execution allowed by rule 'test commands' (CallerInfo=#{caller_module => mod_http_api,ip => {127,0,0,1}})
21:42:29.640 [debug] Executing command mod_admin_extra:set_vcard with Args=[<<"user1">>,<<"localhost">>,<<"FN">>,<<"mi nombre curllll">>]
21:42:29.640 [debug] SQL: "select vcard from vcard where username='user1' and 0=0"
21:42:29.642 [debug] SQL: "begin;"
21:42:29.642 [debug] SQL: "UPDATE vcard SET vcard='<vCard xmlns=''vcard-temp''><FN>mi nombre curllll</FN><N><FAMILY>mi familia11</FAMILY></N><NICKNAME>mi apodoooooooooooooooooooo11</NICKNAME><PHOTO><BINVAL>R0lGODlhDwAPAJECAP//AAAAAP///wAAACH5BAEAAAIALAAAAAAPAA8AAAIulB2Zx5IA4WIhWnnqvQFJDTyhE4khaG5Wqn4tp4ErFnMY+Sll9naUfGpkFL5DAQA7</BINVAL><TYPE>image/gif</TYPE></PHOTO></vCard>' WHERE username='user1'"
21:42:29.644 [debug] SQL: "UPDATE vcard_search SET username='user1', fn='mi nombre curllll', lfn='mi nombre curllll', family='mi familia11', lfamily='mi familia11', given='', lgiven='', middle='', lmiddle='', nickname='mi apodoooooooooooooooooooo11', lnickname='mi apodoooooooooooooooooooo11', bday='', lbday='', ctry='', lctry='', locality='', llocality='', email='', lemail='', orgname='', lorgname='', orgunit='', lorgunit='' WHERE lusername='user1'"
21:42:29.658 [debug] SQL: "commit;"
im using keystone api to create an user (as in Fiware Keystone API Create User).
my steps:
create project with:
curl -s -H "X-Auth-Token:17007fe11124bd71eb60" -H "Content-Type:
application/json" -d '{"tenant": {"description":"Project1",
"name":"proyecto1", "enabled": true}}'
http://localhost:35357/v2.0/tenants -X POST | python
-mjson.tool
create role:
curl -s -H "X-Auth-Token:17007fe11124bd71eb60" -H "Content-Type:
application/json" -d '{"role":{"name":"Project1Admin",
"description":"Role Admin for project1"}}'
http://localhost:35357/v3/roles | python -mjson.tool
Create user:
curl -s -H "X-Auth-Token:17007fe11124bd71eb60" -H "Content-Type:
application/json" -d '{"user": {"default_project_id":
"d0f384973b9f4a57b975fcd9bef10c6e", "description":"admin1",
"enabled":true, "name":"admin", "password":"admin",
"email":"admin#gmail.com"}}' http://localhost:35357/v2.0/users |
python -mjson.tool
last step: create user-role-tenant relationship:
curl -s -H "X-Auth-Token:17007fe11124bd71eb60"
http://localhost:35357/v2.0/tenants/d0f384973b9f4a57b975fcd9bef10c6e/users/admin1/roles/OS-KS/0c10f475076345368724a03ccd1c3403
-X PUT
if i check user:
curl -s -H "X-Auth-Token:17007fe11124bd71eb60" http://localhost:5000/v3/users/admin1 | python -mjson.tool
response:
{
"user": {
"default_project_id": "d0f384973b9f4a57b975fcd9bef10c6e",
"description": "admin1",
"domain_id": "default",
"email": "admin1#gmail.com",
"enabled": true,
"id": "admin1",
"links": {
"self": "http://localhost:5000/v3/users/admin1"
},
"name": "admin1",
"username": null
}
}
I think thats good, But I try to connect with horizon and have an error "Invalid user or password". The result im getting in logs is the following :
keystone.log
2016-04-20 07:56:03.949 2150 WARNING keystone.common.wsgi [-] Could not find user: admin1#gmail.com
2016-04-20 07:56:03.967 2150 INFO eventlet.wsgi.server [-] 127.0.0.1 - - [20/Apr/2016 07:56:03] "HEAD /v3/OS-TWO-FACTOR/two_factor_auth?user_name=admin1%40gmail.com&domain_name=Default HTTP/1.1" 404 159 0.077033
horizon.log:
[Wed Apr 20 07:59:41.934935 2016] [:error] [pid 5963:tid
140154061260544] Login failed for user "admin1#gmail.com".
Anyone knows why this user cant connect with horizon?
thanks
In KeyRock, we use the name field to store the user email, and the username field to store its username. When creating a user, all attributes provided in the request but the name, the username, the default_project_id, the domain_id and the enabled attribute are serialized and stored inside a field called extra. Therefore, your email attribute will be stored in the extra field.
After registering, when loging in to Horizon and providing the user email, Horizon sends a request to Keystone to search for the email in the name field. Since you are entering admin1#gmail.com, but the actual name you provided is admin1, login into Horizon will fail.
Registering the user again with admin1#gmail.com as name (and not email) should fix your problem, but you can also enter admin1 in the email field of the login form if you can't afford to recreate the user.
Hope this solves your issue!