The issue: Webmethods HTTP client is calling the wrong endpoint on my Apache server configured with multiple virtual hosts, based on DNS.
What I think is happening: I think Webmethods HTTP client may be looking up the IP address and using that to perform HTTP operations instead of using the DNS name, which is causing the Apache server to identify it as a request to the main virtual server, not the desired one.
Question: So, how can I make webmethods use the DNS name instead of the IP? Is my theory about the Webmethods HTTP client correct? As far as I can tell this is a very non-standard approach to HTTP Client design.
Here is how it is configured to help you better understand:
Apache ->
host.example.com => /var/www/host/html
host2.example.com => /var/www/host2/html
curl -v http://host.example.com and curl -v http://host2.example.com appropriately return documents from their respective directories.
Configuring pub.client:http with http://host2.example.com causes the webmethods IS server to request http://host.example.com documents (obviously leading to a 404: Not Found).
Note that obviously the system is not returning documents like HTML but rather serving dynamic content.
The comment from Progman is the clue here - basically in order to direct Apache to call your virtual server, the Host header must be given with the expected value. In my example, that would be Host: host2.example.com. I was having webmethods IS copy over the headers exactly like I was posting them from curl, and it was sending Host: localhost:5555 over to my proxied server. I simply created a pipeline Map operation and hardcoded it and it is working fine now.
The oddity to me seems to be that pub.client:http didn't auto-set the Host header for me based on the 'url' value, which is what I would have expected.
Related
I have just exposed my database on openshift and it gives me an 'https://....' url
Does anybody know how to connect using DBeaver by using this url that openshift gave to me.
The error that dbeaver says to me is the following
Malformed database URL, failed to parse the main URL sections.
Short answer: You can't with aRoute
Route can only expose http/https traffic
If you want to expose tcp traffic (like for a database), do not create aRouteand change yourServicetype to "NodePort"`
Check my previous answer for this kind of problem (exposing MQ in this case): How to connect to IBM MQ deployed to OpenShift?
OpenShift doc on NodePorts: https://docs.openshift.com/container-platform/4.7/networking/configuring_ingress_cluster_traffic/configuring-ingress-cluster-traffic-nodeport.html
There's another way to do this.
If your Route is set to "passthrough" it will just look at the SNI headers to determine where to route the traffic but won't unwrap it (and expect http inside) which will let it pass other traffic through to a pod.
I use this mechanism to run a ZNC bouncer (irc traffic) behind SNI.
The downside is you need to provide your own TLS cert inside the pod instead of leveraging the general one available to *.apps.(cluster).com
As for the specific error, "Malformed database URL", I've not used this software but from a quick websearch it looks like you want to rewrite the https://(appname).(clustername).com into a jdbc:.../hostname... string, and then enable TLS in settings.
I found this page that talks about setting it up, so it might be helpful if you've not around found it -- https://github.com/dbeaver/dbeaver/issues/9573
I am currently running a Django site on ec2. The site sends a csv back to the client. The CSV is of varying sizes. If it is small the site works fine and client is able to download the file. However, if the file gets large, I get an ERR_EMPTY_RESPONSE. I am guessing this is because the connection is aborting without giving adequate time for the process to run fully. Is there a way to increase this time span?
Here's what my site is returning to the client.
with open('//home/ubuntu/Fantasy-Fire/website/optimizer/lineups.csv') as myfile:
response = HttpResponse(myfile, content_type='text/csv')
response['Content-Disposition'] = 'attachment; filename=lineups.csv'
return response
Is there some other argument that can allow me to ignore this error and keep generating the file even if it is taking awhile or is large?
I believe that you have any sort of backend proxy server which resets the connection to the Django backend and returns ERR_EMPTY_RESPONSE for the case. You should re-configure timeouts on your backend proxy. Usually that is nginx or apache used as a reverse proxy server.
What is Reverse Proxy Server
A reverse proxy server is an intermediate connection point positioned at a network’s edge. It receives initial HTTP connection requests, acting like the actual endpoint.
Essentially your network’s traffic cop, the reverse proxy serves as a gateway between users and your application origin server. In so doing it handles all policy management and traffic routing.
A reverse proxy operates by:
Receiving a user connection request
Completing a TCP three-way handshake, terminating the initial connection
Connecting with the origin server and forwarding the original request
More info at https://www.imperva.com/learn/performance/reverse-proxy/
One more possible case - your reverse proxy backend server doesn't have enough free space to process response from Django and aborts the request. You can also check free space on your reverse proxy balancer.
Within gunicorn, there is an argument for timeout, -t. When you run gunicorn, the default timeout is 30 seconds. Increase that to something your comfortable with like 90 or 120 seconds, whatever you think fits your application.
I am new to http protocol. When we are sending json message over http to server, How we need to send ?
we need to send the data from different port each time
OR
we can send data form a single port in each time.
If I want to use existing connection to send data in future then whether it is possible or not ?
There is no reason why you would create a TCP socket for each piece of data you want to send — and this has nothing to do with HTTP — and particularly not through a different port each time. In fact, once you hace the socket created and you have connected to the server you should in principle always talk to the server through that socket.
Also, the HTTP protocol uses the port 80, and HTTPS uses 443. That number does not change on demand. Of course you can send HTTP requests through any available port you want and some services even run on special ports using HTTP as the communication protocol but normaly HTTP is 80. See the /etc/services file on linux and read about getaddrinfo().
I'm trying to get an app that uses Server Sent events working on Google Compute Engine, when SSH'd into the box I can view them, but not externally via the ephermeral IP, aka
curl 0.0.0.0/route
works from inside the box but
curl xx.xx.xx.xx/route
just hangs, looking at the headers from other routes there seems to be some sort of cacheing proxy in between the box and the outside word that is preventing server sent events from getting out because the the connection hasn't completed, there is a similar issue with nginx until you set proxy_cache off, but as far as I can tell there is no documentation for configuring the proxy that compute engine uses.
Is it possible to do server sent events from Google Compute Engine and if so what do you have to do to get it to work?
edit:
Request is created with the browser EventSource object, so it has the default headers which look to be Accept:text/event-stream, Cache-Control:no-cache, plus Referer and User-Agent.
The headers I add are Content-Type:text/event-stream, Cache-Control:no-cache, and Connection:keep-alive.
When run in AWS all is fine when I run it behind nginx assuming I modify the config appropriately.
In Google Compute Engine other pages load fine but the route with Server Sent Events just hangs never even receiving headers. The reason I suspect google is sticking a proxy between the GCE box and the outside world is the addition of Via:HTTP/1.1 proxy10205 headers.
There may be magic on the lower network layers but there is no (transparent or otherwise) proxy between your VM and the internet on GCE for the external IP. I'm not sure where the Via header comes from, doesn't the browser/client have a proxy configured?
External IPs are not configured in the most straightforward way on GCE though which might be tripping up something in the stack. I think for external IPs, the external IP itself does not appear anywhere in the VM config, it's translated to the VM internal IP by 1-1 NAT. Loadbalanced IPs do end up on the host with external IP visible though (even though even these are configured in a funny way).
Even though I don't think anything should really care about the server IP for SSE, maybe try setting up a loadbalanced IP pointing to just that one instance and see if it works any better?
"Via:HTTP/1.1 proxy10205" in your HTTP response is not from Google Compute Engine.
The GCE does not strip out the Server-Sent-Events headers. I list the simple steps below which can help you to configure a demo Server-Sent Events on an GCE VM instance:
Create an GCE instance using CentOS image.
Install Apache web server and PHP:
$ sudo yum install httpd php
Create an index.html file with the HTML content from this page :
$ sudo vi /var/www/html/index.html
Create a PHP file called demo_sse.php in the www root directory ($ sudo vi /var/www/html/demo_sse.php ) with the following content:
<?php
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache');
$time = date('r');
echo "data: The server time is: {$time}\n\n";
flush();
?>
Now visit the webpage. You can also verify the header using curl command:
$ curl -H "Accept:text/event-stream" --verbos http://<YOUR-GCE-IP ADDRESS>/demo_sse.php
I'm a bit confused about HTML5 Websockets. I've looked at numerous tutorials out there and a lot of them have different variations of connecting using different ports. What do these ports mean?
Adobe for instance, uses this:
new WebSocket('ws://localhost:1740');
Then another tutorial has this where no ports are required:
new WebSocket("ws://www.websockets.org");
And finally a third tutorial has a port, but it's completely different:
new WebSocket("ws://localhost:8080/echo");
My question would be, why do these vary? How do I know which ports to connect to? Also, I've attempted to do my own connection:
var ws = new WebSocket("ws://test.ontarget-network.com/");
But I get the following error: Unexpected response code: 200
I've tested around and tried connecting to various other "ports" (not knowing what I'm doing obviously, typing in random numbers) and this error would disappear, however, my code
ws.onopen = function(){
alert("Connection Established");
};
would not execute.
I'm trying to fully understand HTML5's Websockets API so I can experiment and create more dynamic applications. Thanks for the help.
The server should have an endpoint that accepts WebSocket connections. So, if that endpoint is /echo you would want to connect to:
ws://localhost:8080/echo/websocket
You will get the Unexpected response code: 200 error if you exclude the /websocket suffix after the endpoint. I was having the same confusion and this link cleared things up a bit for me.
The following comes from the latest WebSocket draft:
By default the WebSocket protocol uses port 80 for regular WebSocket
connections and port 443 for WebSocket connections tunneled over TLS
[RFC2818].
Really though, you should be able to use any valid port not in use. As long as clients are trying to connect to the same port that the server-side script opens for the socket connection, you should be fine.
A quick note on ports:
Port 80 is the HTTP port.
Port 8080 is the alternate HTTP port.
Port 443 is the HTTPS (i.e., HTTP with TLS) port.
Port 1740 in the Adobe code seems like some random port not already in use by other services.
For a full list of preset ports, please see the following:
http://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers
As for your "Unexpected response code: 200" error, I'm guessing that the WebSocket URL you're using on the client side is not pointing to a valid server-side script, but that's hard to comment on without more info.
I had the same issue, But to survive with
Unexpected response code: 200
You need to have either server-side script to handle the web socket, or you can use Node.js to build a you server script.
for the sake of education you can try to biuld your own websocket sever script.
Actually there is something else... You can not open a connection to every port since there is a list of blocked ports in every browser. I remember seeing the full list of ports in 'The tangled Web' from Michal Zalewski; however, I think a quick google will show this also.