My structure
Kubernetes cluster on GKE
Ingress controller deployed using helm
An application which will return list of IP ranges note: it will get updated periodically
curl https://allowed.domain.com
172.30.1.210/32,172.30.2.60/32
Secured application which is not working
What I am trying to do?
Have my clients IPs in my API endpoint which is done
curl https://allowed.domain.com
172.30.1.210/32,172.30.2.60/32
Deploy my example app with ingress so it can pull from the https://allowed.domain.com and allow people to access to the app
What I tried and didn't work?
Deploy the application with include feature of nginx
nginx.ingress.kubernetes.io/configuration-snippet: |
include /tmp/allowed-ips.conf;
deny all;
yes its working but the problem is when /tmp/allowed-ips.conf gets updated the ingress config doesn't
I tried to use if condition to pull the IPs from the endpoint and deny if user is not in the list
nginx.ingress.kubernetes.io/configuration-snippet: |
set $deny_access off;
if ($remote_addr !~ (https://2ce8-73-56-131-204.ngrok.io)) {
set $deny_access on;
}
I am using nginx.ingress.kubernetes.io/whitelist-source-range annotation but that is not what I am looking for
None of the options are working for me.
From the official docs of ingress-nginx controller:
The goal of this Ingress controller is the assembly of a configuration file (nginx.conf). The main implication of this requirement is the need to reload NGINX after any change in the configuration file. Though it is important to note that we don't reload Nginx on changes that impact only an upstream configuration (i.e Endpoints change when you deploy your app)
After the nginx ingress resource was initially created, the ingress controller assembles the nginx.conf file and uses it for routing traffic. Nginx web server does not auto-reload its configuration if the nginx.conf and other config files were changed.
So, you can work around this problem in several ways:
update the k8s ingress resource with new IP addresses and then apply changes to the Kubernetes cluster (kubectl apply / kubectl patch / smth else) / for your options 2 and 3.
run nginx -s reload inside an ingress Pod to reload nginx configuration / for your option 1 with include the allowed list file.
$ kubectl exec ingress-nginx-controller-xxx-xxx -n ingress-nginx -- nginx -s reload
try to write a Lua script (there is a good example for Nginx+Lua+Redis here and here). You should have a good understanding of nginx and lua to estimate if it is worth trying.
Sharing what I implemented at my workplace. We had a managed monitoring tool called Site24x7. The tool pings our server from their VMs with dynamic IPs and we had to automate the whitelisting of the IPs at GKE.
nginx.ingress.kubernetes.io/configuration-snippet allows you to set arbitrary Nginx configurations.
Set up a K8s CronJob resource on the specific namespace.
The CronJob runs a shell script, which
fetches the list of IPs to be allowed (curl, getent, etc.)
generates a set of NGINX configurations (= the value for nginx.ingress.kubernetes.io/configuration-snippet)
runs a kubectl command which overwrites the annotation of the target ingresses.
Example shell/bash script:
#!/bin/bash
site24x7_ip_lookup_url="site24x7.enduserexp.com"
site247_ips=$(getent ahosts $site24x7_ip_lookup_url | awk '{print "allow "$1";"}' | sort -u)
ip_whitelist=$(cat <<-EOT
# ---------- Default whitelist (Static IPs) ----------
# Office
allow vv.xx.yyy.zzz;
# VPN
allow aa.bbb.ccc.ddd;
# ---------- Custom whitelist (Dynamic IPs) ----------
$site247_ips # Here!
deny all;
EOT
)
for target_ingress in $TARGET_INGRESS_NAMES; do
kubectl -n $NAMESPACE annotate ingress/$target_ingress \
--overwrite \
nginx.ingress.kubernetes.io/satisfy="any" \
nginx.ingress.kubernetes.io/configuration-snippet="$ip_whitelist" \
description="*** $(date '+%Y/%m/%d %H:%M:%S') NGINX annotation 'configuration-snippet' updated by cronjob $CRONJOB_NAME ***"
done
The shell/bash script can be stored as ConfigMap to be mounted on the CronJob resource.
Trying to set up a network printer with CUPS.
Followed online documentation that stated:
To discover or share printers using DNS-SD/mDNS, setup .local hostname
resolution with Avahi and restart cups.service.
Followed directions for setting up Avahi to the point where avahi-browse --all --ignore-local --resolve --terminate and avahi-resolve-host-name my-domain.local are both working.
But getent hosts my-domain.local fails to resolve. This results in CUPS failing to print because it can't find my-printer.local.
I read the mdns Github page and saw a note that made me think I didn't need a /etc/mdns.allow file.
nss-mdns has a simple configuration file /etc/mdns.allow for enabling
name lookups via mDNS in other domains than .local.
Note: The "minimal" version of nss-mdns does not read /etc/mdns.allow under any circumstances. It behaves as if the file
does not exist.
In the recommended configuration, no /etc/mdns.allow file is present.
But then I saw the last note in that section:
If, during a request, the system-configured unicast DNS (specified in
/etc/resolv.conf) reports an SOA record for the top-level local name,
the request is rejected. Example: host -t SOA local returns something
other than Host local not found: 3(NXDOMAIN). This is the unicast SOA
heuristic.
I tested that out on my machine and sure enough, I was getting something OTHER than Host local not found....
Adding a /etc/mdns.allow file with a line for .local. and for .local and now I can ping my-printer.local.
I am using postman to test an API I have, all is good when the request does not contain sub-domain, however when I add a sub-domain to URL I am getting this response.
Could not get any response
There was an error connecting to http://subdomain.localhost:port/api/
Why this might have happened:
The server couldn't send a response:Ensure that the backend is working
properly
Self-signed SSL certificates are being blocked:Fix this by turning off
'SSL certificate verification' in Settings > General
Proxy configured incorrectly Ensure that proxy is configured correctly
in Settings > Proxy
Request timeout:Change request timeout in Settings > General
If I copy the same URL from postman and paste it into the browser I get a proper response, is there some kind of configurations I should do to make postman work with sub-domains?
First Go to Settings in Postman:
Off the SSL certificate verification in General Tab:
Off the Global Proxy Configuration and Use System Proxy in Proxy Tab:
Make Request Timeout to 0 (Zero)
Configure Apache:
If the above changes resulted in a 404 response, then continue reading ;-)
Users that host their site locally (like with XAMP and/or WAMP), may be able to visit their virtual sites using https:// prefixed address, but it's a lie, and to really enable SSL (for each virtual-site), configure Apache like:
Open httpd-vhosts.conf file (from Apache's conf/extras directory), in your preferred text editor.
Change the virtual site's settings, into something like:
<VirtualHost *:80 *:443>
ServerName my-site.local
ServerAlias *.my-site.local
DocumentRoot "C:\xampp\htdocs\my-project\public"
SSLEngine on
SSLCertificateFile "path/to/my-generated.cert"
SSLCertificateKeyFile "path/to/my-generated.key"
SetEnv APPLICATION_ENV "development"
<Directory "C:\xampp\htdocs\my-project\public">
Options Indexes FollowSymLinks
AllowOverride All
Order allow, deny
Allow from all
</Directory>
</VirtualHost>
But of course, generate a dummy-SSL-certificate, and change all file paths, like from "path/to/my-generated.cert" into real file addresses.
Finally, test by visiting the local site in the browser, but using http:// (without S) prefixed address; Apache should now give error like:
Bad Request
Your browser sent a request that this server could not understand.
Reason: You're speaking plain HTTP to an SSL-enabled server port.
Instead use the HTTPS scheme to access this URL, please.
I had the same issue. It was caused by a newline at the end of the "Authorization" header's value, which I had set manually by copy-pasting the bearer token (which accidentally contained the newline at its end)
If you get a "Could not get any response" message from Postman native apps while sending your request, open Postman Console (View > Show Postman Console), resend the request and check for any error logs in the console.
Thanks to numaanashraf
Hi This issue is resolved for me.
setting ->general -> Requesttimeout in ms = 0
If all above methods doesn't work check your environment variables, And make sure that the following environments are not set. If those are set and not needed by any other application remove them.
HTTP_PROXY
HTTPS_PROXY
Reference link
For me it was the http://localhost instead of https://localhost.
When getting the following error,
you need to do the following.
Step 1:
In Postman, click the wrench icon, go to settings, then go to the Proxy tab.
Step 2:
Create a custom Proxy. This article explains how to create a custom proxy.
After you create the custom Proxy, make sure you turn the Proxy toggle button to off. I put 61095 in for the proxy server and it worked for me.
Step 3 :
Success
I came up with this solution
In postman go to setting --> proxy
And off Global Proxy Configuration
on the Use System Proxy
And go to windows host configure file
'C:\Windows\System32\drivers\etc\hosts'
Open that file in administrator mode
And add the sub domain to hosts file
For me what worked was to add 127.0.0.1 subdomain.localhost to my host file. On OSX that was /etc/hosts. Not sure why that was necessary as I could reach the subdomain from chrome.
In postman go to setting --> proxy
And off Global Proxy Configuration
For me, it was that route that I was calling in my node server wasn't returning anything. Adding
return res.status(200).json({
message: 'success!',
response: 'success!'
});//
to the route I was calling resolved the issue.
You mentioned you are using a CER certificate.
According to the Postman page on certificates.
Choose your client certificate file in the CRT file field. Currently, we only support the CRT format. Support for other formats (like PFX) will come soon.
The name of the extension CER, CRT doesn't make the certificate that type of certificate but, these are the excepted extensions names.
CER is an X.509 certificate in binary form, DER encoded.
CRT is a binary X.509 certificate, encapsulated in text (base-64) encoding.
You can use OpenSSL to change a CER file into a CRT file. I have not had good luck with it but it looks like this.
openssl x509 -inform PEM -in certificate.cer -out certificate.crt
or
openssl x509 -inform DER -in certificate.cer -out certificate.crt
Postman for Linux Version 6.7.1 - Ubuntu 18.04 - linux 4.15.0-43-generic / x64
I had the same problem and by chance I replaced http://localhost with http://127.0.0.1 and everything worked.
My etc/hosts had the proper entries for localhost and https://localhost requests always worked as expected.
I have no clue why changing localhost for http with 127.0.0.1 solved the issue.
None of these solutions works for me. Postman is not sending any request to the server because postman is not finding the host. So, if you modify your /etc/hosts to
127.0.0.1 localhost
127.0.0.1 subdomain.localhost
It works for me.
For me the issue was that the Content-Length was too big. I placed the content of the body in NotePad++ and counted the characters and put that figure in PostMan and then it worked.
I know it does not directly answer why the op's sub-domain was not working but it might help out someone.
In my case it was invisible spaces that postman didn't recognize, the above string of text renders as without spaces in postman.
I disabled SSL certificate Validation and System Proxy even tried on postman chrome extension(which is about to be deprecated), but when I downloaded and tried Insomnia and it gave those red dots in the place where those spaces were, must have gotten there during copy/paste
For anyone who experienced this issue with real domain instead of localhost and couldn't solve it using ANY OF THE ABOVE solutions.
Try changing your Network DNS (WIFI or LAN) to some other DNS. For me, I used Google DNS 8.8.8.8, 8.8.4.4 and it worked!
solution is very simple if you are using asp.net core 2 application . Inside ConfigureServices method inside startup.cs file all this line
services.AddMvc()
.SetCompatibilityVersion(CompatibilityVersion.Version_2_1)
.AddJsonOptions(x => x.SerializerSettings.ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore);
You just need to turn SSL off to send your request.
Proxy and others come with various errors.
My issue was by putting wrong parameters in the header,
the requested parameters was
Authorization: Token <string>
and is was trying
Authorization Token: <string>
After all the above methods like turning OFF SSL certificate verification, turning ON only Use System Proxy and removing HTTP_PROXY and HTTPS_PROXY system environment variables, it worked.
Note: Had to restart the Postman app, since the environment variables were changed.
Unchecking proxy and SSL Certificate Verification didn't work for me.
Unsetting PROXY environment variables did the trick.
export http_proxy=
export ftp_proxy=
export https_proxy=
Change to the directory where Postman is installed and then:
./Postman
In my case, MVC wasn't able to serialize the results (I accidentally used a model instead of DTO). I debugged down to passing a simple string, which worked. Once I fixed the serialization it all came up.
In my case the (corporate) proxy was using a self-signed SSL certificate which Postman disliked. I discovered it by activating
View->Show Postman console
and retrying the request. The console then showed the certificate error. In
Settings->General
I disabled
SSL certificate verification.
The solution for me, as I'm using the deprecated Postman extension for Chrome, to solve this issue I had to:
Call some GET request using the Chrome Browser itself.
Wait for the error page "Your connection is not private" to appear.
Click on ADVANCED and then proceed to [url] (unsafe) link.
After this, requests through the extension itself should work.
In my case it was a misconfigured subnet. Only one of the 2 subnets in the ELB worked.
I figured this out by doing a nslookup and trying to curl the returned IPs directly. Only one worked.
Postman just kept using the misconfigured one.
I had the same issue.
Turned out my timeout was set too low. I changed it to 30ms thinking it was 30sec. I set it back to 0 and it started working again.
I got the same "Could not get any response" issue because of wrong parameter in header. I fixed it by removing parameter HOST out of header.
PS: Unfortunately, I was pushed to install the other software to get this information. It should be great to get this error message from Postman instead of getting general nonsense.
In my case, I forgot to set the value of the variable in the "CURRENT VALUE" field.
I just experienced this error. In my case, the path was TOO LONG. So url like that gave me this error in postman (fake example)
http://127.0.0.1:5000/api/batch/upload_import_deactivate_from_ready_folder
whereas
http://127.0.0.1:5000/api/batch/upld_impt_deac_ready_folder
worked fine.
Hope it helps someone who by accident read that far...
I built an app on OpenShift Online and now I'm trying to integrate with PayPal. I'm running into SSL cURL errors that I don't know how to address. I've looked through SO, OpenShift Online, PayPal and elsewhere but can't get this issue worked through.
Background:
PHP-based app running on OpenShift Online v2
Setup as
https://*******.rhcloud.com/test/test_IPN.php --- so I can use their
*.rhcloud.com wildcard certificate
Using PayPal "Buy Now" button with PayPal Payments Standard, testing in their sandbox
Using IPN sample code found at
https://github.com/paypal/ipn-code-samples/blob/master/paypal_ipn.php
Here is the portion of the code that seems to be at the root of my problem:
// CONFIG: Please download 'cacert.pem' from "http://curl.haxx.se/docs/caextract.html" and set the directory path
// of the certificate as shown below. Ensure the file is readable by the webserver.
// This is mandatory for some environments.
//$cert = __DIR__ . "./cacert.pem";
//curl_setopt($ch, CURLOPT_CAINFO, $cert);
Problem:
[1] using code "as is" (lines 79-80 commented out) throws curl error: "SSL connect error"
[2] using lines 79-80 uncommented out (and cacert.pem placed in same dir as php script) throws curl error: "Problem with the SSL CA cert (path? access rights?)"
It's likely I'm missing something simple here. Any help getting this to work properly on OpenShift Online is greatly appreciated. Thanks!
This line is pretty suspect:
$cert = __DIR__ . "./cacert.pem";
Basically you would end up with something like $cert equaling /home/path./cacert.pem, which I am pretty sure is not what you want, and why you are getting the ssl error, it can't find the certificate.
That could be corrected to:
$cert = __DIR__ . "/cacert.pem";
It also might be better to store the cacert.pem in your $OPENSHIFT_DATA_DIR and reference it as such:
$cert = getenv("OPENSHIFT_DATA_DIR")."cacert.pem";
And make sure that the permissions on the cacert.pem are at least 0644
chmod 0644 $OPENSHIFT_DATA_DIR/cacert.pem
Solution:
Force the use of TLS 1.2
Commenting out lines 79-80 and adding
curl_setopt($ch, CURLOPT_SSLVERSION, 6); // Force TLS 1.2
did the trick for me. Hope this helps someone else.
P.S. The need for TLS 1.2 came from this PayPal article https://www.paypal-knowledge.com/infocenter/index?page=content&widgetview=true&id=FAQ1914&viewlocale=en_US
I have a Mercurial repository running on Scm-manager proxied behind Nginx. A variety of smaller repositories run fine, so the basic setup seems OK.
Additionally, this same box runs Owncloud. I've tweaked the client_max_body_size on the server to 1000M so large files can be transferred. This works, and I have a variety of large files syncing between the server and clients.
However, when I try pushing a large Mercurial repository for the first time (1007 commits vs. about 80 for the other largest on this system) I get the following:
abort: HTTP Error 413: FULL head
Everything I've read about 413 errors doesn't seem to apply. First, it recommends setting the body size, which I've stated is already at 1G. Next, this seems to apply that the header is too large, which makes sense given that it's probably trying to check 1000+ revisions in the remote repository.
Another thing I've encountered is large_client_header_buffers. I've set this to insanely huge values like "64 128k" on both the server and http levels (read something about it not working on servers) but that didn't change anything.
I also looked at the scm-manager logs but see nothing, so this seems to stop with Nginx.
Thoughts? Here is part of my Nginx server configuration:
server {
server_name thewordnerd.info;
listen 443 ssl;
ssl_certificate /etc/ssl/certs/thewordnerd.info.crt;
ssl_certificate_key /etc/ssl/private/thewordnerd.info.key;
root /srv/www/thewordnerd.info/public;
client_max_body_size 1000M;
location /scm {
proxy_pass http://127.0.0.1:8080/scm;
include /etc/nginx/proxy_params;
}
}
The problem is the header buffer of the application server, this is because of mercurial uses very big headers. You have to increase the size of the header buffer and this application server specific. In case you are using the standalone version, you have to edit the server-config.xml and increase the requestHeaderSize value.
replace:
<Set name="requestHeaderSize">16384</Set>
with:
<Set name="requestHeaderSize">32768</Set>
Source: https://groups.google.com/forum/#!topic/scmmanager/Afad4zXSx78
I had HTTP Error: 413 (Request Entity Too Large) on my attempt to push. Resolved by adding client_max_body_size 2M; to /etc/nginx/nginx.conf. Wondering if maybe 1000M doesn't exceed the client_max_body_size...