We have enabled ACL's and TLS for Consul cluster in our environment. We have disabled the UI as well. But when I use the URL: http://<consul_agent>:8500/v1/coordinate/datacenters. How can disable the URL's as this?
I tested with adding the following to the consulConfig.json:
"ports":{
"http": -1
}
this did not solve the problem.
Apart from the suggestion provided to use "http_config": { "block_endpoints": I am trying to use the ACL Policy if that can solve.
I enabled the ACL's first
I created a policy using the command: consul acl policy create -name "urlblock" -description "Url Block Policy" -rules #service_block.hcl -token <tokenvalue>
contents of the service_block.hcl: service_prefix "/v1/status/leader" { policy = "deny" }
I created a agent token for this using the command: consul acl token create -description "Block Policy Token" -policy-name "urlblock" -token <tokenvalue>
I copied the agent token from the output of the above command and pasted that in the consul_config.json file in the acl -> tokens section as "tokens": { "agent": "<agenttokenvalue>"}
I restarted the consul agents (did the same in the consul client also).
Still I am able to access the endpoint /v1/status/leader. Any ideas as what is wrong with this approach?
That configuration should properly disable the HTTP server. I was able to validate this works using the following config with Consul 1.9.5.
Disabling Consul's HTTP server
Create config.json in the agent's configuration directory which completely disables the HTTP API port.
config.json
{
"ports": {
"http": -1
}
}
Start the Consul agent
$ consul agent -dev -config-file=config.json
==> Starting Consul agent...
Version: '1.9.5'
Node ID: 'ed7f0050-8191-999c-a53f-9ac48fd03f7e'
Node name: 'b1000.local'
Datacenter: 'dc1' (Segment: '<all>')
Server: true (Bootstrap: false)
Client Addr: [127.0.0.1] (HTTP: -1, HTTPS: -1, gRPC: 8502, DNS: 8600)
Cluster Addr: 127.0.0.1 (LAN: 8301, WAN: 8302)
Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false, Auto-Encrypt-TLS: false
==> Log data will now stream in as it occurs:
...
Note the HTTP port is set to "-1" on the Client Addr line. The port is now inaccessible.
Test connectivity to HTTP API
$ curl localhost:8500
curl: (7) Failed to connect to localhost port 8500: Connection refused
Blocking access to specific API endpoints
Alternatively you can block access to specific API endpoints, without completely disabling the HTTP API, by using the http_config.block_endpoints configuration option.
For example:
Create a config named block-endpoints.json
{
"http_config": {
"block_endpoints": [
"/v1/catalog/datacenters",
"/v1/coordinate/datacenters",
"/v1/status/leader",
"/v1/status/peers"
]
}
}
Start Consul with this config
consul agent -dev -config-file=block-endpoints.json
==> Starting Consul agent...
Version: '1.9.5'
Node ID: '8ff15668-8624-47b5-6e83-7a8bfd715a56'
Node name: 'b1000.local'
Datacenter: 'dc1' (Segment: '<all>')
Server: true (Bootstrap: false)
Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, gRPC: 8502, DNS: 8600)
Cluster Addr: 127.0.0.1 (LAN: 8301, WAN: 8302)
Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false, Auto-Encrypt-TLS: false
==> Log data will now stream in as it occurs:
...
In this example, the HTTP API is enabled and listening on port 8500.
Test connectivity to HTTP API
If you issue a request to one of the blocked endpoints, the following error will be returned.
$ curl localhost:8500/v1/status/peers
Endpoint is blocked by agent configuration
However, access to other endpoints are still permitted.
$ curl localhost:8500/v1/agent/members
[
{
"Name": "b1000.local",
"Addr": "127.0.0.1",
"Port": 8301,
"Tags": {
"acls": "0",
"build": "1.9.5:3c1c2267",
"dc": "dc1",
"ft_fs": "1",
"ft_si": "1",
"id": "6d157a1b-c893-3903-9037-2e2bd0e6f973",
"port": "8300",
"raft_vsn": "3",
"role": "consul",
"segment": "",
"vsn": "2",
"vsn_max": "3",
"vsn_min": "2",
"wan_join_port": "8302"
},
"Status": 1,
"ProtocolMin": 1,
"ProtocolMax": 5,
"ProtocolCur": 2,
"DelegateMin": 2,
"DelegateMax": 5,
"DelegateCur": 4
}
]
Related
I tried to make some http requests through GCP's Cloud Function with Python 3.10 runtime, some went well, and some went wrong.
To find out the reason, I ping the IP of each url, and the IP I need has no response:
ping ip(173.194.217.106) of url(https://www.google.com): True
ping ip(69.147.92.11) of url(https://www.yahoo.com): True
ping ip(13.107.42.14) of url(https://www.linkedin.com): True
ping ip(117.56.7.114) of url(https://data.moi.gov.tw): False
Is there any way to make a successful request to https://data.moi.gov.tw through Cloud Function?
Here are the materials to reproduce the results with Cloud Function (Gen1):
main.py:
import platform # For getting the operating system name
import subprocess # For executing a shell command
import requests
def ping(host):
"""
Returns True if host (str) responds to a ping request.
Remember that a host may not respond to a ping (ICMP) request even if the host name is valid.
"""
# Option for the number of packets as a function of
param = '-n' if platform.system().lower()=='windows' else '-c'
# Building the command. Ex: "ping -c 1 google.com"
command = ['ping', param, '1', host]
return subprocess.call(command) == 0
def main(event):
d_ip_url = {
'173.194.217.106' : 'https://www.google.com',
'69.147.92.11' : 'https://www.yahoo.com',
'13.107.42.14' : 'https://www.linkedin.com',
'117.56.7.114' : 'https://data.moi.gov.tw',
}
for ip, url in d_ip_url.items():
print(f'ping ip({ip}) of url({url}):', ping(ip))
requirements.txt:
# Function dependencies, for example:
# package>=version
requests
The cloud Function settings:
{
"name": "projects/corgis-361708/locations/asia-east1/functions/ping-test",
"httpsTrigger": {
"url": "https://asia-east1-corgis-361708.cloudfunctions.net/ping-test",
"securityLevel": "SECURE_ALWAYS"
},
"status": "ACTIVE",
"entryPoint": "main",
"timeout": "60s",
"availableMemoryMb": 256,
"serviceAccountEmail": "corgis-361708#appspot.gserviceaccount.com",
"updateTime": "2022-09-21T06:04:31.746Z",
"versionId": "2",
"labels": {
"deployment-tool": "console-cloud"
},
"sourceUploadUrl": "https://storage.googleapis.com/uploads-918581105162.asia-east1.cloudfunctions.appspot.com/78ad8f77-d16c-412c-843f-51238703fbbf.zip",
"runtime": "python310",
"maxInstances": 1,
"ingressSettings": "ALLOW_ALL",
"buildId": "0c8bf5d0-3467-4516-8fea-c39d0e093c2e",
"buildName": "projects/647355426154/locations/asia-east1/builds/0c8bf5d0-3467-4516-8fea-c39d0e093c2e",
"dockerRegistry": "CONTAINER_REGISTRY"
}
Finally deployed a self-hosted Sourcegraph, v3.39.1. I'm running on Google Compute Engine on a Container Optimized OS VM. Got everything working except I'm having difficulty getting SMTP set up though Google Workspace's smtp-relay. When I run a sendTestEmail command, I get back an opaque error:
{
"data": {
"sendTestEmail": "Failed to send test email: EOF"
}
}
Here is a redacted snippet of my config:
"email.address": "myemailaddress#mydomain.com",
"email.smtp": {
"authentication": "PLAIN",
"domain": "mydomain.com",
"username": "myemailaddress#mydomain.com",
"password": "REDACTED",
"host": "smtp-relay.gmail.com",
"port": 587
}
For a quick smoke test, I ran netcat (nc smtp-relay.gmail.com 587) from the container and did sent EHLO mydomain.com. The smtp-relay responded with a "at your service" so I assume that worked. Any one have any other tips? Any logs I can check? (I didn't see anything obvious, but I'm new to Sourcegraph)
I have deployed an API-Platform app using JWT token to ElasticBeanstalk which, as usual, works fine in my local server.
On EB though it is denying access to logged in users despite the correct BearerToken being provided.
This is the error thrown:
{
"errors": [
{
"message": "Access Denied.",
"extensions": {
"category": "graphql"
},
"locations": [
{
"line": 6,
"column": 9
}
],
"path": [
"retrievedQueryUser"
]
}
],
"data": {
"retrievedQueryUser": null
}
}
The query in question attempts to retrieve user profile info through the below graphql config:
* "retrievedQuery"={
* "item_query"=UserProfileResolver::class,
* "normalization_context"={"groups"={"get-owner"}},
* "security"="is_granted('IS_AUTHENTICATED_FULLY') and object == user"
* },
So, it should be a simple matter of checking if the users IS_AUTHENTICATED_FULLY and if it is the user him/herself trying to execute the query.
Far as I could tell, by dump below on /vendor/symfony/security-core/Authorization/AuthorizationChecker.php, it's failing to retrieve a token.
var_dump($this->tokenStorage->getToken()->getUser()->getUsername());
I did a cursory comparison of phpinfo() between my local installation and the one at AWS-EB and could not find any obvious mismatch.
This is the config for JWT at /config/packages/lexik_jwt_authentication.yaml.
lexik_jwt_authentication:
secret_key: '%env(resolve:JWT_SECRET_KEY)%'
public_key: '%env(resolve:JWT_PUBLIC_KEY)%'
pass_phrase: '%env(JWT_PASSPHRASE)%'
user_identity_field: email
token_ttl: 1800
Just to confirm that the users are able to login. It's passing through the isGranted() check that fails.
Any ideas?
EDIT - add `/config/packages/security.yaml
security:
# https://symfony.com/doc/current/security.html#where-do-users-come-from-user-providers
encoders:
App\Entity\User:
algorithm: auto
#algorithm: bcrypt
#algorithm: argon2i
cost: 12
providers:
database:
entity:
class: App\Entity\User
property: email
firewalls:
dev:
pattern: ^/(_(profiler|wdt)|css|images|js)/
security: false
refresh:
pattern: ^/api/token/refresh
stateless: true
anonymous: true
api:
pattern: ^/api
stateless: true
anonymous: true
json_login:
check_path: /api/login_check
success_handler: lexik_jwt_authentication.handler.authentication_success
failure_handler: lexik_jwt_authentication.handler.authentication_failure
guard:
authenticators:
- app.google_login_authenticator
- App\Security\TokenAuthenticator
entry_point: App\Security\TokenAuthenticator
user_checker: App\Security\UserEnabledChecker
access_control:
- { path: ^/login, roles: IS_AUTHENTICATED_ANONYMOUSLY }
- { path: ^/admin, roles: ROLE_SUPERADMIN }
- { path: ^/api/token/refresh, roles: IS_AUTHENTICATED_ANONYMOUSLY }
- { path: ^/api, roles: IS_AUTHENTICATED_ANONYMOUSLY }
role_hierarchy:
ROLE_PROVIDER: ROLE_USER
ROLE_ADMIN: [ROLE_PROVIDER, ROLE_EDITOR]
ROLE_SUPERADMIN: ROLE_ADMIN
Upon further research I found out that Apache was stripping the authorization token from the request.
On the method supports of /lexik/jwt-authenticator-bundle/Security/Guard/JWTTokenAuthenticator, the dump as below will not include the token on AWS:
var_dump($request->headers->all());
var_dump($_SERVER);
As per this question, this is an issue of Apache configuration which is not accepting the authorization headers.
The indicated solution is to add the following to .htaccess:
SetEnvIf Authorization "(.*)" HTTP_AUTHORIZATION=$1
This resolves the issue, though one should note that the local Apache installation works fine without the above edit to .htaccess.
So, it should also be possible to change Apache config directly, but I could not find how to go about it.
EDIT: Later I found a specific instruction on 'JWT-Token' docs as follows, that confirm that solution on this link.
I installed a KMS container on my server and I downloaded kurento hello world java application on my server but when I go to my java web application using my server IP adress I have to remote stream and and the following error (in firefox):
ICE failed, see about:webrtc for more details
in the about:webrtc It tells me that there is no STUN and no TURN server specified (and a lot of following output not very clear to me) The problem is that I specified a STUN server on the WebRtcEndpoint.conf.ini.
Here is my docker-compose.yml file:
kurento:
image: fiware/stream-oriented-kurento:latest
volumes:
- ./kurento.conf.json:/etc/kurento/kurento.conf.json:ro
- ./defaultCertificate.pem:/etc/kurento/defaultCertificate.pem:ro
- ./WebRtcEndpoint.conf.ini:/etc/kurento/modules/kurento/WebRtcEndpoint$
ports:
- "8888:8888"
- "8433:8433"
here is my kurento.conf.json file:
{
"mediaServer" : {
"resources": {
// //Resources usage limit for raising an exception when an object creatio$
// "exceptionLimit": "0.8",
// // Resources usage limit for restarting the server when no objects are $
// "killLimit": "0.7",
// Garbage collector period in seconds
"garbageCollectorPeriod": 240
},
"net" : {
"websocket": {
"port": 8888,
"secure": {
"port": 8433,
"certificate": "defaultCertificate.pem",
"password": ""
},
//"registrar": {
// "address": "ws://localhost:9090",
// "localAddress": "localhost"
//},
"path": "kurento",
"threads": 10
}
}
}
}
and my WebRtcEndpoint.conf.ini
; Only IP address are supported, not domain names for addresses
; You have to find a valid stun server. You can check if it works
; usin this tool:
; http://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/
stunServerAddress=62.71.2.168
stunServerPort=3478
; turnURL gives the necessary info to configure TURN for WebRTC.
; 'address' must be an IP (not a domain).
; 'transport' is optional (UDP by default).
; turnURL=user:password#address:port(?transport=[udp|tcp|tls])
;pemCertificate is deprecated. Please use pemCertificateRSA instead
;pemCertificate=<path>
;pemCertificateRSA=<path>
;pemCertificateECDSA=<path>
and the certificate has been generated with :
certtool --generate-privkey --outfile defaultCertificate.pem
echo 'organization = your organization name' > certtool.tmpl
certtool --generate-self-signed --load-privkey defaultCertificate.pem \
--template certtool.tmpl >> defaultCertificate.pem
sudo chown kurento defaultCertificate.pem
and I went on my https://localhost:8433/kurento to validate the certificate
When I start the kurento container with docker-compose up I can see on the logs that my conf. file has been loaded:
kurento_1 | "websocket":
kurento_1 | {
kurento_1 | "port": "8888",
kurento_1 | "secure":
kurento_1 | {
kurento_1 | "port": "8433",
kurento_1 | "certificate":
"defaultCertificate.pem",
kurento_1 | "password": ""
kurento_1 | },
kurento_1 | "path": "kurento",
kurento_1 | "threads": "10"
kurento_1 | }
.....
kurento_1 | "WebRtcEndpoint":
kurento_1 | {
kurento_1 | "stunServerAddress": "62.71.2.168",
kurento_1 | "stunServerPort": "3478",
kurento_1 | "configPath":
"\/etc\/kurento\/modules\/kurento"
kurento_1 | },
and I start the hello world example with :
sudo mvn compile exec:java -Dkms.url=wss://localhost:8433/kurento
at this point everything seems to work OK, no error output.
When I try to access my web application from a client with https://:8443 the web page is loaded correctly and can start the stream. But I have no remote stream and have the error I printed at the beginning.
UPDATE 1
I changed the version of the kurento image in docker-compose.yml from
image: fiware/stream-oriented-kurento:latest
to:
image: fiware/stream-oriented-kurento:6.6.0
And now it is working sometimes. I have the same error (ICE failed, see about:webrtc for more details) but if I reload the page multiple time, it end up working after some reload. Any suggestion about what I am doing wrong?
UPDATE 2
I realized that when the web application start working (after multiple reload), the next time I access the web applicaiton, it will always work, until I restart the KMS. Then I have to reaload the page multiple time again to have the remote stream.
Now that I realized that, I tried again with image: fiware/stream-oriented-kurento:latest and it has the exact same behavior. I have to reload multiple time the page to make it work. I have no clue why is that, any idea?
Looking into your problem I feel the ICE candidates have not been created properly on both sides.
Have you configured the STUN / TURN in the webApplication (JS)?
If you haven't modify the example I believe they are not configured by default
Check into the options.configuration. Example:
var options = {
........ some options here ....
configuration: {
iceServers:[{
"url": "turn:xxx.xxx.xxx:port",
"username": "xxxxxx",
"credential": "xxxxxx"
}]
}
webRtcPeer = new kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv(options,<callback-here>);
Can you provide logs for both the Firefox ICE candidates generation and KMS ICE generation?
In addition is KMS up and running in the same machine as the tutorial?
More information about kurento_utils used in the hello-world: http://doc-kurento.readthedocs.io/en/stable/mastering/kurento_utils_js.html#using-data-channels
General example of WebRTC configuration the STUN on the Web client side:
https://www.w3.org/TR/webrtc/#simple-peer-to-peer-example
I want to integrate cosmos-auth with Idm GE.
Config for node.js application is:
{
"host": "192.168.4.180",
"port": 13000,
"private_key_file": "key.pem",
"certificate_file": "cert.pem",
"idm": {
"host": "192.168.4.33",
"port": "443",
"path": "/oauth2/token"
},
"cosmos_app": {
"client_id": "0434fdf60897479588c3c31cfc957b6d",
"client_secret": "a7c3540aa5de4de3a0b1c52a606b82df"
},
"log": {
"file_name": "/var/log/cosmos/cosmos-auth/cosmos-auth.log",
"date_pattern": ".dd-MM-yyyy"
}
}
When i send HTTP POST request directly to IDM GE to url
https://192.168.4.33:443/oauth2/token
with required parameters i get ok results:
{
access_token: "LyZT5DRGSn0F8IKqYU8EmRFTLo1iPJ"
token_type: "Bearer"
expires_in: 3600
refresh_token: "XiyfKCHrIVyludabjaCyGqVsTkx8Sf"
}
But when i curl the cosmos-auth node.js application
curl -X POST "https://192.168.4.180:13000/cosmos-auth/v1/token" -H
"Content-Type: application/x-www-form-urlencoded" -d
"grant_type=password&username=idm&password=idm" -k
I get next result:
{"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred"}
Has anyone encountered something similar?
What could be the problem?
The error i made was using unsigned certificate.How clumsy of me.
So either sign the certificate or insert additional element in options object (rejectUnauthorized: false)
var options = {
host : host,
port : port,
path : path,
method : method,
headers: headers,
rejectUnauthorized: false
};
or in the beginning of the file insert:
process.env.NODE_TLS_REJECT_UNAUTHORIZED = '0';
Ofcourse this is only temporary solution until we use fully signed cert.
Anyways error handling and logs in cosmos-auth node.js app should show a little bit more.