I have successful login to a webpage by the script below but the webpage stuck at the redirect page. I has no idea how to pass through it. Below is my code
use strict;
use warnings;
use LWP::UserAgent;
use HTTP::Request::Common qw(POST);
use HTTP::Cookies;
my $URL="http://www.redirect.com";
my $UA = LWP::UserAgent->new();
$UA->ssl_opts( verify_hostnames => 0 );
my $req =HTTP::Request::Common::POST("$URL",
Content_type=>'form-data',
Content =>[
'username'=>'name',
'password'=>'pass',
]
);
my $resp=$UA->request($req);
if ($resp->is_success) {
my $res2 = $UA->post($resp->base, []);
open(OUTFILE1, ">html1.txt");
print OUTFILE1 $res2->decoded_content;
if ($res2->is_success) {
if( ($resp->code() >= 200) && ($resp->code() <400) ) {
open(OUTFILE, ">html.txt");
binmode(OUTFILE, ":utf8");
print OUTFILE $resp->decoded_content;
}else{
print "Error: ". $resp->status_line. "\n";
}
}
}
This is the part of the output file i get
<HTML>
<HEAD>
<TITLE>
</TITLE>
</HEAD>
<BODY onLoad="document.AUTOSUBMIT.submit();">This page is used to hold your data while you are being authorized for your request.<BR>
<BR>You will be forwarded to continue the authorization process. If this does not happen automatically, please click the Continue button below.
<FORM NAME="AUTOSUBMIT" METHOD="POST" ENCTYPE="application/x-www-form-urlencoded" <INPUT TYPE="SUBMIT" VALUE="Continue"></FORM>
</BODY>
</HTML>
How to pass through this auto direct page to reach the website i want? Have modified according to the answer but nothing was output.
Edit 8/7/2017
I try the way simbabque suggest and debug by print out the $res2 in html1.txt. The output look like below
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved here.</p>
<hr>
<address>Apache/2.2.3 (CentOS) Server at www Port 80</address>
</body></html>
I have no idea what this mean. Can anyone help?
The interesting part about this is probably the headers. A normal website would issue a redirect status code, like a 302 Found. But that's clearly not the case here, or they wouldn't need this rather strange HTML thing.
The HTML has built-in JavaScript execution.
<BODY onLoad="document.AUTOSUBMIT.submit();">
It tells the browser to submit the form directly when the page has loaded. Your problem is that LWP::UserAgent cannot do that, because it doesn't have JS support.
But since this always happens, it's trivial to code around it. All you need to do is submit that form every time you log in.
my $res = $ua->request($req);
if ($res->is_success) {
my $res2 = $ua->post($res->base, []);
if ($res2->is_success) {
...
}
}
The form has no parameters. The only <input> element there is is the submit button, and since that doesn't have a name attribute it does not show up as a parameter. The URL is likely the same as the one you submitted originally, but it might have done a real redirect already, so it's better to use the base attribute of the response object.
I do wonder why they make this process so weird. It certainly doesn't authorize anything. It might set additional cookies, like one of those marketing redirect thingies, but that's not visible from what you've showed. And it does not stop automation either.
Related
I'm trying to send a 0 or a 1 to a database within my InfluxDB instance via a POST request from an HTML form. I've done this successfully lots of times through curl, but I can't make it work with a simple HTML form. Consider this HTML code:
<!doctype html>
<!-- this file is called like http://my.influx.server/my_page_name.html -->
<html>
<head>
<title>my simple html/influx sender</title>
<meta charset="UTF-8"/>
</head>
<body>
<form action="http://my.influx.server:8086/write?db=db_name" method="post" enctype="text/plain">
<input name="data" type="hidden" value="my_measurement,tag_name=stuff value=1"/>
<input type="submit" value="insert 1"/>
</form>
<form action="http://my.influx.server:8086/write?db=db_name" method="post" enctype="text/plain">
<input name="data" type="hidden" value="my_measurement,tag_name=stuff value=0"/>
<input type="submit" value="insert 0"/>
</form>
</body>
</html>
The curl command for sending a 1 would be like:
curl -i -XPOST 'http://my.influx.server:8086/write?db=mydb' --data-binary 'my_measurement,tag_name=stuff value=1'
So I tried to make a simple HTML form with just 2 buttons. The code above is the closest I could get to at least try to process the "line interface" syntax, however I'm getting either an error message or just no response and I don't get anything in my InfluxDB. The error message from the code above is:
unable to parse 'data=my_measurement,tag_name=stuff value=1\r': invalid number
If you have a close look at the end of the string, you see a \r that obviously gets added and I suspect that this breaks number parsing (I had something similar some time ago), but at least this seems to try to evaluate the line at all. However, I haven't found a way to remove or avoid the \r. Has someone an idea how to achieve this?
Also, please consider the following additional information:
I want it really simple, just a small HTML file with possibly a bit of JavaScript code, but I'd really like to avoid using PHP, jQuery and such. Also, I'm trying to get used to HTML5 as you might notice, but this shouldn't be the problem.
In this case, I don't need a timestamp for each key press, so instead of passing a timestamp I just use the current time. This is achieved by omitting the timestamp, so the string excluding the \r should be syntactically correct.
I also looked for alternatives, however there was only the idea to use JSON and this seems not to be supported any more due to performance reasons (which I wouldn't expect in my case).
The curl command uses the --data-binary parameter, but it seems I don't have anything like this in HTML. I'm aware of binary enctypes like application/x-binary, but they don't work, because they URL-encode the string and this won't pass the syntax check. The only enctype I found that worked at least close enough is text/plain.
I'm also aware of form data not being sent, if the corresponding <input> element has no name attribute. Then I noticed that the curl string was built like my_measurement,tag_name=stuff value=1, possibly multiple such lines separated by \n, which is not like POST key-value-pairs as in a=1&b=2 (i. e. there is no key, that would be the name attribute). Trying to trick it with name="my_measurement,tag_name" and value="stuff value=1" (which would resemble the original string) was not successful and I still couldn't figure out, which key is expected. I tried with content, query etc. and ended up using data. I kept this then because in the docs they talk about "data" and none of the keys made any difference, as long as one is provided. I suspect InfluxDB to just use the first POST variable ignoring the name, but I can't find any clear statement on this.
I also tried several invisible <input> types like just hidden or a regular textbox hidden by style. This made no difference. Neither did visible elements.
I also considered using AJAX, but I couldn't find anything useful about binary POSTs without key-value content. I even would cope with a page that only works e. g. for Firefox for now, so I don't need to switch between different AJAX object creation algorithms and such (yes, I know, jQuery helps, but see first point above).
EDIT 1: I tried to reproduce the error with curl:
curl -i -XPOST 'http://my.influx.server:8086/write?db=home' --data-binary 'my_measurement,tag_name=stuff value=1\r'
This led to the error message:
unable to parse 'my_measurement,tag_name=stuff value=1\\r': invalid number
with headers:
HTTP/1.1 400 Bad Request
Content-Type: application/json
Request-Id: ...
X-Influxdb-Build: OSS
X-Influxdb-Error: unable to parse 'my_measurement,tag_name=stuff value=1\r': invalid number
X-Influxdb-Version: 1.7.9
X-Request-Id: ...
Date: ...
Content-Length: 78
I conclude:
\r seems to be differently encoded in the error message (characters \ and r instead of an actual carriage return), but in the header it's only \r, however it doesn't make a difference regarding the parsing error, so this is comparable.
There is obviously no key name involved, so this is still different from my attempt above.
EDIT 2: I found out how to show the request headers from a call to curl. The command is:
curl -v -XPOST 'http://my.influx.server:8086/write?db=db_name' --data-binary 'my_measurement,tag_name=stuff value=1'
The relevant portion of the output of the command is:
> POST /write?db=db_name HTTP/1.1
> Host: my.influx.server:8086
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Length: 37
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 37 out of 37 bytes
< HTTP/1.1 204 No Content
< Content-Type: application/json
< Request-Id: ...
< X-Influxdb-Build: OSS
< X-Influxdb-Version: 1.7.9
< X-Request-Id: ...
< Date: Sat, 25 Jan 2020 10:54:11 GMT
I conclude:
Content type of the request invoked by curl with --binary-data is application/x-www-form-urlencoded.
Unfortunately I couldn't achieve to see the actual request body, so I'll try again with some URL-encoded variants. However, my_measurement,tag_name=stuff value=1 is 37 characters as in the request header, so I assume there is no key name like data involved. Currently, I get the same error message I had before I posted this question: unable to parse 'data=my_measurement%2Ctag_name%3Dstuff+value%3D1': missing fields
The \r is gone, but I still can't send data without a key name and the whole string is invalid due to URL-encoding. How to get rid of the URL-encoding?
Finally, I found a solution with JavaScript that worked. This Mozilla doc page was the key to a POST form without keys. My HTML page now looks like this:
<!doctype html>
<!-- this file is called like http://my.influx.server/my_page_name.html -->
<html>
<head>
<title>my simple html/influx sender</title>
<meta charset="UTF-8"/>
</head>
<body>
<form id="form1">
<button>insert 1</button>
</form>
<form id="form0">
<button>insert 0</button>
</form>
<script>
function sendData(value)
{
const str = "my_measurement,tag_name=stuff value=" + value;
const xhr = new XMLHttpRequest();
xhr.addEventListener("load", function(event) {
alert("Success");
});
xhr.addEventListener("error", function(event) {
alert("Error");
});
xhr.open("POST", "http://my.influx.server:8086/write?db=db_name");
xhr.setRequestHeader("Content-Type", "application/x-www-form-urlencoded");
xhr.send(str);
}
const form0 = document.getElementById("form0");
const form1 = document.getElementById("form1");
form0.addEventListener("submit", function(event) {
event.preventDefault();
sendData(0);
});
form1.addEventListener("submit", function(event) {
event.preventDefault();
sendData(1);
});
</script>
</body>
</html>
Note the stripped-down form definitions: There are no actions, methods or enctypes any more, as they are set via JavaScript. Also, there is no regular submit element, instead it is a regular button, however I don't know if this is needed. I'll investigate that later.
The main part is in the script tag underneath the forms. A function sendData prepares an XMLHttpRequest object for POSTing a prepared string and invokes its send method. This function is used in the submit events of each form. Also, this function registers event handlers for successful and failed requests.
The lines below the sendData function identify the forms and register event listeners on their submit event. Each listener prevents its form from submitting in a regular fashion and invokes the appropriate sendData call instead, which will successfully insert values into InfluxDB.
Be aware, though, there is still no guarantee to detect every error. I tried to insert a string into an integer field, which failed, but I still got the "Success"-alert. I'm going to investigate that later.
So in total, I see this problem as sufficiently resolved for my purposes and I hope this helps anyone stumbling across it.
This was a pretty useful post, I ran into this issue with the Sigfox backend and callbacks.
If you put an & at the end of the URL and use content type text/plain the \r\n issue is solved.
I'm trying to understand the behavior of innerHTML in the code below. I want to permanently add a new div block every time I hit the button, but it seems that the new block only pops up for a split second then disappears.
Does anyone know why this is the case, and how to fix it?
Also, when I change the code to use appendChild instead of innerHTML, I get an error saying Argument 1 of Node.appendChild is not an object.. I'm not sure what this means.
Any help is much appreciated!
Below is the code:
<DOCTYPE html>
<html>
<body>
<form onSubmit="loadData()">
<input type="submit" id="button">
</form>
<div id="block">List of items:</div>
<script>
function loadData(){
document.getElementById("block").innerHTML += "<div>item</div>";
// document.getElementById("block").appendChild("<div>item</div>");
};
</script>
</body>
</html>
Because you are submitting then the page reloads and your HTML is obliterated.
If you need items to persist then you will need to use cookies, localStorage or a server-side solution.
function addItem()
{
document.getElementById("block").innerHTML += "<div>item</div>";
}
<DOCTYPE html>
<html>
<body>
<form>
<input type="button" id="button" onclick="addItem()" />
</form>
<div id="block">List of items:</div>
</body>
</html>
you are submitting the page. appendChild or innerHtml happen directly after submit, before the new page is loaded. once the new page is loaded, the current page (with the applied modifications) is dismissed and replaced with the new page.
if you wanted something to happen on the new page, you would need to execute the code on that page. (or don't use a form submit, but rather some ajax for sending the form).
The reason why appendChild is not working for you, is that appendChild expects a dom node as parameter, not a string. it would be like document.getElementById("foo).appendChild(document.createElement("div")). (the tricky part is that with createElement you get an empty element, you would also need to put the content you want into it.
Your first question is already answered by #lee.
Your problem with your second answer is, that you can not use appendChild like you did. If u want to use append child, according to the mozilla developer docs you will have to to something like this:
var mydiv = document.createElement("div");
mydiv.appendChild(document.createTextNode("item"));
document.getElementById("block").appendChild(mydiv);
to get the result you asked for.
I am currently working on a semi dynamic page where I use some GET functionality to personalize it. I also echo the date a couple of places. At the bottom of this page, I would like to have a button that gives the visitor the option to download/open this page as PDF. Without the header. I have integrated DOMPDF, but I simply cant get it to work properly and need some help. I have tried a couple of things found here on Stackoverflow, with no success.
In basic, I need the whole page printed in the PDF, but it should not open when page is loaded. But triggered by the button. And then without the header (one spesific div). Is this possible?
<?php
require_once("dompdf/dompdf_config.inc.php");
$html =
'<html><body>'.
'<p>This is a test for '.
'<?php echo htmlentities(substr(urldecode($_SERVER["QUERY_STRING"]), 1)); ?> </p>'.
'<p>Thank you for reading.</p>'.
'</body></html>';
$dompdf = new DOMPDF();
$dompdf->load_html($html);
$dompdf->render();
$dompdf->stream("the_inquiry.pdf");
return true;
?>
<html>
<body>
<div class="shouldnotPrintToPDF">
Content.
</div>
<div class="shouldPrintToPDF">
Sensitive content.
Open or save as PDF
</div>
</body>
</html>
This is basically our one page presentation. And contains a lot of text, so I will not present all of that here. But in this way, I have to write the page twice, in $html = as well as inside the actual tag. And the PDF save/open option pops up right from the start, which it should not. I also wish to append the echo htmlentities part to the actual pdf-name.. is this possible? The PDF opens and contains what put into the $html = just fine. But not triggered by the link.
Update:
When i do exactly what you perform here, I get an error "The requested URL /inquiry.php&pdf=1 was not found on this server." I have the page I am trying to print in pdf on root level, but the DOMPDF is in /dompdf.. I dont know if that has anything to do with it?
Update:
When i edited the link, i get all this information up in a new page, like below.
[_parse_properties(margin:=1.2cm)(empty)_parse_properties]
[_parse_sections[section[_parse_properties(display:=-dompdf-page)
(counter-reset:=page)(empty)_parse_properties]#html#section]
[section[_parse_properties(empty)_parse_properties]#empty#section]
_parse_sections][_parse_sections[section[_parse_properties(display:=block)
(empty)_parse_properties]#div##map##dt##isindex#section]
[section[_parse_properties(empty)_parse_properties]#empty#section]
_parse_sections][_parse_sections[section[_parse_properties(page-break
-before:=avoid)(display:=block)(counter-increment:=page)
(empty)_parse_properties]#body#section]
[section[_parse_properties(empty)_parse_properties]#empty#section]_parse_sections]
[_parse_sections[section[_parse_properties(display:=block)(margin:=1em
0)(empty)_parse_properties]#p##dl##multicol#section]
[section[_parse_properties(empty)_parse_properties]#empty#section]_parse_sections]
[_parse_sections[section[_parse_properties(display:=block)(margin-left:=40px)
(empty)_parse_properties]#dd#section]
[section[_parse_properties(empty)_parse_properties]#empty#section]_parse_sections]
[_parse_sections[section[_parse_properties(display:=block)(margin:=1em
40px)(empty)_parse_properties]#blockquote#section]
[section[_parse_properties(empty)_parse_properties]#empty#section]_parse_sections]
[_parse_sections[section[_parse_properties(display:=block)(font-style:=italic)
(empty)_parse_properties]#address#section]
[section[_parse_properties(empty)_parse_properties]#empty#section]_parse_sections]
[_parse_sections[section[_parse_properties(display:=block)(text-align:=center)
(empty)_parse_properties]#center#section]
Do you have any idea what it might be caused by?
Breakthrough:
When I activated DOMPDF_DPI it actually opens as PDF, but now all of the text comes on first line of the second page of the PDF. Like, all the text comes out on top of each other. Also, when it opens the PDF, the ?&pdf=1 are included in the htmlentities query string, which looks very messy since it is supposed to be a personalized page as well as the PDF.
You can set dompdf to parse CSS #media queries for standard media types (screen, print, voice, etc.). By default dompdf parses the "screen" media type styles, but you can change this in the configuration file. See the DOMPDF_DEFAULT_MEDIA_TYPE configuration setting. Then you just need to pass the original URL querystring with an appended variable telling the page to render to PDF. If your original URL is something like the_inquiry.php?name=Joe then your PDF URL could be the_inquiry.php?name=Joe&pdf=1.
Your code would then look similar to the following:
<?php
ob_start();
?>
<html>
<head>
<style>
#media print {
.shouldnotPrintToPDF, .pdflink { display: none; }
}
</style>
</head>
<body>
<div class="shouldnotPrintToPDF">
Content.
</div>
<div class="shouldPrintToPDF">
Sensitive content.
Open or save as PDF
</div>
</body>
</html>
<?php
if ( isset( $_GET['pdf'] ) ) {
require_once 'dompdf/dompdf_config.inc.php';
$html = ob_get_clean();
$dompdf = new DOMPDF();
$dompdf->load_html($html);
$dompdf->render();
$dompdf->stream("the_inquiry.pdf");
}
?>
Example:
<form>
<input type='submit'>
</form>
When submitted results in:
http://example.com/?
How to make it:
http://example.com/
?
[This is a very simple example of the problem, the actual form has many fields, but some are disabled at times. When all are disabled, the trailing ? appears]
In my case I'm using window.location, not sure it's the best alternative, but it's the only one I could make it work:
$('#myform').submit(function()
{
... if all parameters are empty
window.location = this.action;
return false;
});
My real use was to convert GET parameter to real url paths, so here is the full code:
$('#myform').submit(function()
{
var form = $(this),
paths = [];
// get paths
form.find('select').each(function()
{
var self = $(this),
value = self.val();
if (value)
paths[paths.length] = value;
// always disable to prevent edge cases
self.prop('disabled', true);
});
if (paths.length)
this.action += paths.join('/')+'/';
window.location = this.action;
return false;
});
Without using Javascript, I'm not sure there is one. One way to alleviate the problem may be to create a hidden input that just holds some junk value that you can ignore on the other side like this:
<input type="hidden" name="foo" value="bar" />
That way you will never have an empty GET request.
This is an old post, but hey.. here ya go
if you are using something like PHP you could submit the form to a "proxy" page that redirects the header to a specific location + the query.
For example:
HTML:
<form action="proxy.php" method="get">
<input type="text" name="txtquery" />
<input type="button" id="btnSubmit" />
</form>
PHP (proxy.php)
<?php
if(isset($_GET['txtquery']))
$query = $_GET['txtquery'];
header("Location /yourpage/{$query}");
?>
I am assuming this it what you are trying to do
I was looking for similar answer. What I ended up doing was creating a button that redirects to a certain page when clicked.
Example:
<button type="button" value="Play as guest!" title="Play as guest!" onclick="location.href='/play'">Play as guest!</button>
This is not an "answer" to your question but might be a good work around. I hope this helps.
Another option would be to check the FormData with javascript before submitting.
var myNeatForm = document.getElementById("id_of_form");
var formData = new FormData(myNeatForm); // Very nice browser support: https://developer.mozilla.org/en-US/docs/Web/API/FormData
console.log(Array.from(formData.entries())); // Should show you an array of the data the form would be submitting.
// Put the following inside an event listener for your form's submit button.
if (Array.from(formData.entries()).length > 0) {
dealTypesForm.submit(); // We've got parameters - submit them!
} else {
window.location = myNeatForm.action; // No parameters here - just go to the page normally.
}
I know this is a super old question, but I came across the same issue today. I would approach this from a different angle and my thinking is that in this day and age you should probably be using POST rather than GET in your forms, because passing around values in a querystring isn't great for security and GDPR. We have ended with a lot of issues where various tracking scripts have been picking up the querystring (with PII in the parameters), breaking whatever terms of services they have.
By posting, you will always get the "clean url", and you won't need to make any modifications to the form submit script. You might however need to change whatever is receiving the form input if it is expecting a GET.
You will get a trailing question mark when submitting an empty form, if your server adding trailing slash to URL and your action URL of form - is directory (and not file) and:
Trailing slash in the action attribute URL (action="/path/").
With dot (with or without trailing slash after it) instead specific URL (action="." or action="./").
With empty action (action="").
Form without action attribute.
Try to specify an action-URL without trailing slash:
action="path"
or
action="./path/sub"
and
action="/path"
or
action="/path/sub"
I'm running Firefox 2.0.0.14.
I have a form on a webpage which is working fine with the GET method.
I'm using a plugin to view my browser's HTTP request when submitting the form, and here it is :
GET /postComment.php?review=2&comment=Testing HTTP/1.1
...
However, if I make the simple change from method=GET to method=POST on the form:
GET /postComment.php HTTP/1.1
...
It isn't even attempting to POST.
Any possible reasons for this, under any circumstances?
EDIT: Here is the form:
<form method=POST action="postComment.php"><input type=hidden name=review value="2"><input type=submit value="Postit">
</form>
Is the action parameter of the form tag set? Could Javascript be intercepting the post? Some HTML from your form would be helpful, or an example link :)
I'm guessing your plugin is not capturing the POST variables. Since the output of your plugin is:
GET /postComment.php HTTP/1.1
How are you catpuring your POST varables? $_POST['key'] or $_REQUEST['key'] should contain your value if the form action and method are set properly.
POST will not be found in the query string.
EDIT:
if you are trying to capture the post value, you can check it with something like this:
if (isset($_REQUEST['submit'])) {
echo $_REQUEST['review'];
}
or
if (isset($_POST['submit'])) {
echo $_POST['review'];
}
Acorn
I would start by making sure your HTML is valid XHTML. Wrap attribute values in quotations and end the input elements with />. Use a valid DOCTYPE.
Also, try changing the value of the submit button to "submit" (as that is the default).
Try it out in different browsers, including the latest version of Firefox.
Firstly, your <form> tag needs to be adjusted:
<form method="post" ... >
Secondly, I have a function called debugArray that I use to spit out misbehaving arrays. It's very handy:
function debugArray($array){
echo("<pre>");
print_r($array);
echo("</pre>");
}
Then, call it in your code like this:
debugArray($_POST);
By looking at the entire contents of the $_POST array, you can see exactly what is being sent, what is not, and how it is being sent.
I'm willing to wager that one of the following is true:
You have a spelling mistake in a field name, remembering that names are case sensetive.
Your form field is outside your <form> tags.
You have a value that is not being escaped correctly, or otherwise being dropped from the $_POST for whatever reason.
Edit: And I would also be inclined to update your copy of Firefox.
I was having the same problem, till I remembered that my .htaccess file hides my PHP extension, and for reasons that someone else can explain (tech stuff) All I did was remove the .php extensión in the action property and it worked.
So, I went from:
action="folder/folder/file.php"
To:
action="folder/folder/file"
And the print_r($_POST) displayed the full array
I really Hope this helps someone else with the same problem.
And if anyone can technically explain why this is happening, it would be very educational 🙃