Here is a wget command that executes a HTML / PHP stack report suite that is hosted by a third party - we don't have control over the PHP or HTML page
wget --no-check-certificate --http-user=/myacc --http-password=mypass -O /tmp/myoutput.csv "https://myserver.mydomain.com/mymodule.php?myrepcode=9999&action=exportcsv&admin=myappuserid&password=myappuserpass&startdate=2011-01-16&enddate=2011-01-16&reportby=mypreferredview"
All the elements are working perfectly:
--http-user / --http-pass as offered by a browsers standard popup for username and password prompt
-O /tmp/myoutput.csv - the output file of interest
https://myserver.mydomain.com/mymodule.php?myrepcode=9999&action=exportcsv&admin=myappuserid&password=myappuserpass&startdate=2011-01-16&enddate=2011-01-16&reportby=mypreferredview"
The file generated on the fly by the parameters
myrepcode=9999 - a reference to the report in question
action=exportcsv internally written in the function
admin=myappuserid the third party operats SSL to access the site - then internal username and password stored in a database to access the functions of the site)
password=myappuserpass
startdate=2011-01-16 this and end data are parameters specific to the report 9999
enddate=2011-01-16
reportby=mypreferredview This is an option in the report that facilitates different levels of detail or aggregation
The problem is that the reportby parameter is a radio button selection in a list of 5 selections (sure I enough the default is highest level of aggregation , I want the last one which is the most detailed)
Here is a sample of the HTML page code for the options of reportby
The tags in the HTML are not whitelisted - so I will send the sample if requested
<td>View by</td>
<td>
<input class="naf-radio" name="reportby" id="reportby[thedefault]" value="thedefault" type="radio">The Default
<input class="naf-radio" name="reportby" id="reportby[myleastpreferred]" value="myleastpreferred" type="radio">My Least Preferred
<input class="naf-radio" name="reportby" id="reportby[mysecondleastpreferred]" value="mysecondleastpreferred" type="radio">My Second Least Preferred
<input class="naf-radio" name="reportby" id="reportby[mythirdleastpreferred]" value="mythirdleastpreferred" type="radio">My Third Least Preferred
<input class="naf-radio" name="reportby" id="reportby[mypreferred]" value="mypreferred" type="radio">My Preferred
</td>
No matter which of the reportby items I select in the wget statement - thedefault is always executed.
Questions
1) Has anyone come across this notation in HTML (id=inputname[inputelement])
I spoke to a senior web developer and he has never seen this notation for inputs (id=inputname[inputelement]) - and w3schools do not appear familiar with this either based on an extensive search
2) Can a wget command select a none default radio item when executing the command ?
This probably will be initially received with a "Use CURL" response- however the wget approach works very well in the limited environment I am operating in - particularly as I need to download 10000 of these such items.
Thanks ahead of response
A radio button is just another form element and can be passed through the querystring usually. Some applications will demand that parameters are passed as POST data but this isn't that common in my experience.
What you'll need to do is find the name of the radio buttonand the value that is on the desired option. You then just add &name=value to your current url and it should act like selecting that radio button.
The notation id=inputname[inputelement] could exist in javascript or similar languages but not in HTML. In HTML its just name=value type of declarations in urls (and attribtues and other things). In this case I would assume that this string was meant to have been interpreted in whatever code generated the HTML rather than rendered to screen.
Also you need to make sure you urlencode any values that you are putting in the url to make sure they don't contain any illegal characters (eg an & or = will confuse it completely).
If the querystring method doesn't work then wget has a --post-data switch that allows you to specify data to be posted whcih is what a form would do. If you use --post-data=reportby=mypreferred I hope you should have more success with that.
If this still fails then I would use some tool to view your wget request as well as your request through a browser and compare the headers and data to see what is different about them. one such tool for this is fiddler (http://www.fiddler2.com/fiddler2/) though I'm sure many others exist.
Related
guys! I'm looking for a solution or some ideas on how to solve my task.
There is a video surveillance camera(vendor: Hikvision) with an accessible web-interface.
In the web-interface, there is a field Device Name containing data I need to retrieve by means of the Zabbix server and further to use this data for renaming discovered hosts.
Since Hikvision cameras support SNMP, I've tried the SNMP agent in Zabbix. I turned out that Hikvision MIB doesn't contain data from that field.
Also exploring web-interface through Developer tools in Google Chrome I stumbled upon the string Request URL: http://10.90.187.16/ISAPI/System/deviceInfo which gives such response in XML format:
<DeviceInfo xmlns="http://www.hikvision.com/ver20/XMLSchema" version="2.0">
<deviceName>1.5.1.1</deviceName>
<deviceID>566eec0b-6580-11b3-81a1-1868cb48861f</deviceID>
<deviceDescription>IPCamera</deviceDescription>
<deviceLocation>hangzhou</deviceLocation>
<systemContact>Hikvision.China</systemContact>
<model>DS-2CD2155FWD-IS</model>
<serialNumber>DS-2CD2155FWD-IS20170417AAWR749464587</serialNumber>
<macAddress>18:68:cb:48:86:1f</macAddress>
<firmwareVersion>V5.4.5</firmwareVersion>
<firmwareReleasedDate>build 170124</firmwareReleasedDate>
<encoderVersion>V7.3</encoderVersion>
<encoderReleasedDate>build 170123</encoderReleasedDate>
<bootVersion>V1.3.4</bootVersion>
<bootReleasedDate>100316</bootReleasedDate>
<hardwareVersion>0x0</hardwareVersion>
<deviceType>IPCamera</deviceType>
<telecontrolID>88</telecontrolID>
<supportBeep>false</supportBeep>
<supportVideoLoss>false</supportVideoLoss>
</DeviceInfo>
Where the tag <deviceName>1.5.1.1</deviceName> contains required data and now the question is how to put two and two together by means of Zabbix.
Digging into Zabbix documentation I've found an article about creating an Item based on HTTP agent with XML request . Unfortunately there are not any exmaples how to do it exactly.
Has somebody had such experience? Any clues will be helpful
You can create an HTTP Agent item, set it to TEXT type and point it to http://10.90.187.16/ISAPI/System/deviceInfo (don't forget the authentication, if required!), Zabbix will retrieve the full XML.
To get the desired value you have to create a dependent item, point it to the previous item and set up a preprocessing step.
Create a single XML Xpath preprocessing rule with parameter string(/DeviceInfo/DeviceName) to get the 1.5.1.1 value
If you want to get the firmware version, create another dependent item and set up the XPath to string(/DeviceInfo/FirmwareVersion) and so on for every element you need.
If you want a single value you can use a single item, adding the preprocessing rule to the http agent item. I use my solution for flexibility, maybe one day I'll need another XML element or maybe a firmware update will add some element to the page.
Dependent items are more flexible, but of course the full XML uses more storage in the database for stuff you don't need right now: it's a tradeoff, either way works!
Currently, I use a single SSM parameter to store a set of properties separated by newlines, like this:
property1=value1
property2=value2
property3=value3
(I am aware of the 4K size limit, it's fine.)
This works well, for normal String type parameters that store non-sensitive information like environment configuration, but I'd also like to do similar for secrets using the SecureString parameter type.
The problem is that I can't edit the parameter value in the console because it's using a HTML input field of type="password" that doesn't handle newlines.
The multi-line value works fine with the actual parameter store backend - I can set a value with multiple lines with the SSM API no problem and they can be read with the EC2 CLI properly too.
But I can't edit them using the console. This is a problem because the whole point of using a SecureString parameter is that I intend the only place to edit/view these secrets to be via the console (so that permissions are controlled and access is audited).
There's a few infrastructure workarounds I could implement (one parameter for each secret, store the secrets on S3 or other secret storing service, etc.) but they all have drawbacks - I'm just trying to find out if there's a way around this using the console?
Is there any way I can work around this and use the console to edit multi-line SecureString parameters?
Any kind of browser workaround or hack that I might be able to use to tell the browser to use a textarea instead of a "password" type field?
I'm using Chrome, but I'd be happy to work around this by using another browser or something (editing the secrets is pretty rare, and viewing multi-line values in the console works fine).
EDIT
After posting this question, AWS notified me there was a whole new "AWS Systems Manager" UI, but it still has the same problem - I tried the below browser hacks on this new UI, but no luck.
Failed browser hack attempt 1: I tried opening the browser console, running document.getElementById("Value").value = "value1\nvalue2" and then clicking the save button, which set the value I injectec, but the newline was filtered out.
Failed browser hack attempt 2: I tried using the browser instpector to change the element to a TextArea and then typed in two lines of input and clicked save, but that didn't set the value at all.
From https://docs.aws.amazon.com/cli/latest/userguide/cli-using-param.html#cli-using-param-file, I learned you can pass a file as parameter to the --value argument. So if your file is called secrets.properties, you can do this:
aws ssm put-parameter --type SecureString --name secrets --value file://secrets.properties
I found a way to do it, but it's too much effort and too weird - if anyone can find a simpler way, I will mark that as the answer.
The hacky workaround is to install the "Tamper Chrome" extension + app, then capture the XHR request as the browser sends it and edit the new lines into the JSON.
Blech. Plus "Tamper Chrome" is pretty awful, I don't want to run it on my machine.
This might be better to use the new secrets manager that was launched recently. The interface for it is very close to parameter store but it has better support for multiple parameters in one place.
I wonder if the change in the console was due to the expected release of the service since they have a pricing model around secrets whereas parameter store is free
In the end, I decided the answer to this question is "don't do that". Not that I would've wanted to hear that when I was trying to make it work.
You should use a separate SSM param per secret for these reasons:
ability to grant permissions at fine grained level; e.g. you have an API password for calling your service, and a DB password for the service talk to a DB - if you store them in the same secret you couldn't only grant access to the API password.
ability to track key access separately - the SSM access logs can only tell you that the target machine/user accessed the SSM param at that time, it won't be able to tell you which secret was accessed
ability to use separate KMS keys to encrypt
Just watch out for the fact that you can only request a max of 10 SSM params at a time.
if you want, you can try with my app https://github.com/ledongthuc/awssecretsmanagerui
I try to create it to easier to update multi-line values and binary easier. Hope it's helpful with your case.
I logged in today to make another logic app today, and i noticed the return output for (in this example) a Http-call has changed.
Before, i have a memory of the whole object showing in the output of an action in the workflow. Now i just se this:
Picture below:
The output body is only a string in some kind of encryption...
Does the Workflow definition Language where i want to reach one specific value in the Json-body still work? Or was this Update a major overhall.
I'm lost here.
Does the Workflow definition Language where i want to reach one specific value in the Json-body still work?
Yes, We can still do it, seems like you at the designer over view of a run, if you want to see full of inputs and output click on the Run details tab on top of this screen.
enter image description here
The content is base64 encoded, if you want to decode with in the logic apps for any other action input, we can use the base64 function which are explained here (https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-workflow-definition-language#functions)
Issue:
An authentication server (not under my control) is stripping get values after the first from the returnUrl. Is there any way to work around this? (Or around the larger problem as a whole?)
e.g. http://authentication.corporate.com?returnUrl=http://localserver/addcomment.php?FirstName=Sally&SecondName=Sparrow redirects as http://authentication.corporate.com?returnUrl=http://localserver/addcomment.php?FirstName=Sally, stripping any subsequent $_GET values.
(I am led to believe through my research that the stripping of the $_GET values may be an issue with what the authentication server is doing, but I have no way of getting access to it. If this is the case, is there a workaround? If this is not the case, what am I doing wrong?)
Context:
I am writing part of a web application which is to allow users to add comments to documents. The web application must retrieve the user's (correct) corporate username, first name and last name.
I am updating a previous version of this web application which allowed users to add comments in a two-step process.
Step 1 After clicking an 'Add Comment' hyperlink, the user is authenticated and a returnUrl value directs them to a page where they may add their comment. http://authentication.corporate.com?returnUrl=http://localserver/addcomment.php
Step 2 In addcomment.php $_POST values have been retrieved (and confirmed as correct) from the authentication server. The user may then enter their comments in a textarea and submit them via action=post to a final page which inserts the comment and user information into a database.
I would like to reduce this to a one-step process where the user may type in a comment on the main page. The way I am attempting to do this is by passing the comment as a $_GET value to be returned via the authentication server. e.g. http://authentication.corporate.com?returnUrl=http://localserver/addcomment.php?FirstName=Sally&SecondName=Sparrow along with the $_POST values.
Is there a way to get the Last-Modified-Date of a file on a Web Site?
i.e. Here is an example file I have out there:
http://www.ymcadetroit.org/atf/cf/%7B2101903E-A11A-4532-A64D-9D823368A605%7D/Birmingham_Youth_Sports_Parent_Manual.pdf
Go to the website you want to know about, wait for it to fully load, then go to the address bar and write this:
javascript:alert(document.lastModified)
You'll get a popup that says when it was last modified.
The HTTP intends the Last-Modified header field to declare the last modification date. But the server needs to know that date.
On static files whose content is sent directly to the client and not interpreted otherwise by the server (e.g. .html, .css, .js) it uses the last modified date of that file. But on files that generated content dynamically (PHP, Python, etc.) the script needs to specify that information itself. But unfortunatly many scripts don’t to that.
So if a Last-Modified header field is present, you can use that information. But if not, you cannot determin the last modification date.
Here is some C# code to do it:
public DateTime GetLastModifyTime(string url)
{
WebRequest request = WebRequest.Create(url);
request.Credentials = CredentialCache.DefaultNetworkCredentials;
request.Method = "HEAD";
using (WebResponse response = request.GetResponse())
{
string lastModifyString = response.Headers.Get("Last-Modified");
DateTime remoteTime;
if (DateTime.TryParse(lastModifyString, out remoteTime))
{
return remoteTime;
}
return DateTime.MinValue;
}
}
I realize this question is 4 years old, but a search of the web proved that satisfactory answers remain rare. Peter's answer is part of the solution. When I had the same problem to solve, that got me started. But the rest of the solution...
As he said, the web server must be configured to send the last-modified date ... so how do you configure the web server?
Assuming you have the necessary level of control, you first need to enable server side includes. There are several ways to do this - one of which is the "xbithack". A good reference is http://httpd.apache.org/docs/current/howto/ssi.html.
Assuming you've done this, you need to set the execute bit on any html file that needs to have server-side includes parsed. This can be done at the command line of a UNIX-like system: chmod u+x file.html or on the Mac using get-info (command-I) on the file.
This leaves the snippet to actually put in your file, which looks like this:
This document last modified <!--#flastmod file="index.html" -->
Since I found many, many recommendations that didn't include this, and simply used the javascript document.lastModified, I suspect that some servers give you what you want with the javascript version, whereas some (including the one hosting our stuff) don't.
To obtain the last modified date from client side, you can access the HTML DOM using the lastModified property using JavaScript.
The lastModified property grabs the information from the head portion sent with all web requests. The value can be manually set by developers on the web-server side of things so it may not reflect the actual last modified date of the file responsible for delivering the content.
Example
<!DOCTYPE html>
<html>
<body>
<b>document.lastModified : </b>
<script>document.write( document.lastModified );</script>
</body>
</html>
The specific command in JavaScript that retrieves this is document.lastModified and can easily be converted into a Date object as follows :
var x = new Date(document.lastModified);
More information can be found on the site I used as a reference w3 schools : HTML DOM lastModified Property
I believe the web server must be configured to send the last-modified date in an HTTP-header, this is certainly one way. Check out section 14.29 Last-Modified of this document:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
You can do the following to get Last-Modified:
https://superuser.com/a/991895
Using curl:
curl -s -v -X HEAD http://foo.com/bar/baz.pdf 2>&1 | grep '^< Last-Modified:'
Using wget:
wget --server-response --spider http://example.com/bar/example.pdf 2>&1 | grep -i Last-Modified
With just plain HTML, no you cannot.
You can with PHP, or ASP, or any other server side language.
I'm not an expert in headers, but believe you are looking for this:
There is a way to check the date when a file was modified:
View HTTP headers in Google Chrome?
Check in there (Chrome's Developer Tools / Network / Selected File / Headers) the "If-Modified-Since" variable.
Until now this has helped me to achieve what you are asking, get a file's modification date.
In php:
print getlastmod();
print gmdate('D, d M Y H:i:s', getlastmod());