I would love to get some help in logging into the Fidelity website and navigate within it. My attempts so far have not led me to anywhere significant. So here is the code that I have written, after much consultation with answers around the web. The steps are:
Login to Fidelity
Check if response code not 200, but is 302 or 303 and my code passes this test (with a code of 302).
Then I check the number of cookies returned (there were 5) and for each cookie I try to navigate to a different web page within Fidelity (I do this five times, once for each cookie, simply because I do not know which subscript "j" of the variable "cookie" will work).
function loginToFidelity(){
var url = "https://www.fidelity.com";
var payload = {
"username":"*********",
"password":"*********"
};
var opt = {
"payload":payload,"method":"post","followRedirects" : false
};
var response = UrlFetchApp.fetch(encodeURI(url),opt);
if ( response.getResponseCode() == 200 ) {
Logger.log("Couldn't login.");
return
}
else if (response.getResponseCode() == 303 || response.getResponseCode() == 302) {
Logger.log("Logged in successfully. " + response.getResponseCode());
var cookie = response.getAllHeaders()['Set-Cookie']
for (j = 0; j < cookie.length; j++) {
var downloadPage = UrlFetchApp.fetch("https://oltx.fidelity.com/ftgw/fbc/oftop/portfolio#activity",
{"Cookie" : cookie[j],"method" : "post","followRedirects" : false,"payload":payload});
Logger.log(downloadPage.getResponseCode())
Logger.log(downloadPage.getContentText())
}
}
}
For each choice of the subscript "j", I get the same answer for the ResponseCode (always 302) as well as the same answer for ContentText. The answer for ContentText is obviously incorrect as it is not what it is supposed to be. The ContentText is shown below:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved here.</p>
</body></html>
Based on this, I have two questions:
Have I logged into the Fidelity site correctly? If not, why do I get a response code of 302 in the login process? What do I need to do differently to login correctly?
Why am I getting such a strange and obviously incorrect answer for my ContentText while getting a perfectly reasonable ResponseCode of 302? What do I need to do differently, so that I can get the password-controlled page within Fidelity, whose url is "https://oltx.fidelity.com/ftgw/fbc/oftop/portfolio#activity"?
NOTE: Some other tests have been done in addition to the one stated above. Results from these tests are provided in the discussion below.
Here is something which worked for me. You may have found the solution already, not sure. Remember to fill in your loginid where the XXXX is and the pin number for YYYY.
I understand this is python code, not the google script, but you get the idea about the code flow.
import requests, sys, lxml.html
s = requests.Session()
r = s.get('https://login.fidelity.com')
payload = {
'DEVICE_PRINT' : 'version%3D3.5.2_2%26pm_fpua%3Dmozilla%2F5.0+(x11%3B+linux+x86_64%3B+rv%3A41.0)+gecko%2F20100101+firefox%2F41.0%7C5.0+(X11)%7CLinux+x86_64',
'SavedIdInd' : 'N',
'SSN' : 'XXXXX',
'PIN' : 'YYYYY'
}
r = s.post(login_url, data=payload, headers=dict(referer='https://login.fidelity.com'))
response = s.get('https://oltx.fidelity.com/ftgw/fbc/oftop/portfolio')
print response.content
mwahal, you left out the critical form action url (your login_url is undefined)
this works (if added to your python code)
login_url = 'https://login.fidelity.com/ftgw/Fas/Fidelity/RtlCust/Login/Response/dj.chf.ra'
btw here's the result of the print after the post showing successful login
{"status":
{
"result": "success",
"nextStep": "Finish",
"context": "RtlCust"
}
}
or adding some code:
if r.status_code == requests.codes.ok:
status = r.json().get('status')
print(status["result"])
gets you "success"
Unfortunately the answer from #mwahal doesn't work anymore - I've been trying to figure out why, will update if I do. One issue is that the login page now requires a cookie from the cfa.fidelity.com domain, which only gets set when one of the linked JavaScript files is loaded.
One alternative is to use selenium, if you just want to navigate the site, or seleniumrequests if you want to tap into Fidelity's internal APIs.
There is a hitch with seleniumreqeusts for the transactions API... the API requires Content-Type: application/json and seleniumrequests doesn't seem to support custom headers in requests. So I use selenium to log in, call one of the APIs that doesn't need that header, copy then edit the response's request header, and use regular requests to get the transactions:
from seleniumrequests import Chrome
import requests
# Log into Fidelity
driver = Chrome()
driver.get("https://www.fidelity.com")
driver.find_element_by_id("userId-input").send_keys(username)
driver.find_element_by_name("PIN").send_keys(password)
driver.find_element_by_id("fs-login-button").click()
r = driver.request('GET', 'https://digital.fidelity.com/ftgw/digital/rsc/api/profile-data')
headers = r.request.headers
headers['accept'] = "application/json, text/plain, */*"
headers['content-type'] = "application/json"
payload = '{"acctDetails":[{"acctNum":"<AcctId>"}],"searchCriteriaDetail":{"txnFromDate":1583639342,"txnToDate":1591411742}}'
api = "https://digital.fidelity.com/ftgw/digital/dc-history/api"
r = requests.post(api, headers=headers, data=payload)
transactions = r.json()
Related
I'm having issues lately with my web app written in Google Apps Script.. From time to time it doesn't load properly and returns a 409 response.. It seems as it has no real reason, it's randomly. If I try to reload the page sometimes it loads properly, sometimes returns 409. The issue also just occurs some days and times. I can't figure out what can be the reason, the only thing I've noticed is that when the issue occurs also Google Apps Script Program seem to be slow / failing (for example takes longer than usual to save a file, sometimes fails on saving or showing logs etc).
Screenshot from web inspector when the issue occurs..
If the issue is on Google's end on their servers I guess there's not much I can do, but still checking if anyone else experienced these issues? Or is there anything I can do?
Also want to make sure it's not in my code.. I'm not in anyway a pro when it comes to programming, but a happy amateur. But if it was in my code, at least in my mind it would fail every time and not just only sometimes? I googles a bit and found on this site that when making POST requests you need to have your parameters correct, I'm using Twilio in my script but it's in a separate function and is not initiated when the script loads and I have it written as the linked documentation suggests..
Inserting some code snippets here - if anyone sees something that looks weird or could be the reason for my issues I would much appreciate your help!
SMS Function - can this be the reason??
function sendSms_(to, body) {
var messages_url = "https://api.twilio.com/2010-04-01/Accounts/ACe5***************/Messages.json";
var payload = {
"To": to,
"Body" : body,
"From" : "Killnoise"
};
var options = {
"method" : "post",
"contentType": "application/x-www-form-urlencoded", //This line I added after reading the linked documentation
"payload" : payload
};
options.headers = {
"Authorization" : "Basic " + Utilities.base64Encode("AC********************:f8be***************")
};
try { UrlFetchApp.fetch(messages_url, options); } catch(Err) {}
return;
}
doGet function - anything wrong here??
function doGet(e){
var staffId = e.parameter.staffId;
var key = e.parameter.key;
var lang = e.parameter.lang
// staffId = "S102", key = "abc"
if(auth_(staffId, key, lang) == true){
var output = HtmlService.createTemplateFromFile('StaffHtml').getRawContent();
/*/
Am I doing right above and below when the output is returned? I'm a bit green
here - should I write this in another way? Not seeing this could be the reason
for my current issue though..
/*/
output = output.replace('{{selectAlert}}', language('selectAlert', lang));
output = output.replace('{{takenAlert}}', language('takenAlert', lang));
output = output.replace('{{declineGivenAlert}}', language('declineGivenAlert', lang));
output = output.replace('{{beta}}', language('beta', lang));
output = output.replace(/{{surgePrompt}}/g, language('surgePrompt', lang));
output = output.replace('<%staffId%>', lang + staffId);
output = output.replace('{{invitationSent}}', language('invitationSent', lang));
output = output.replace('{{timeOutAlert}}', language('timeOutAlert', lang));
output = output.replace('{{newNotifications}}', language('newNotifications', lang));
var out = HtmlService.createHtmlOutput(output)
.addMetaTag('viewport', 'width=device-width, initial-scale = 1, user-scalable = no')
.setTitle('Killnoise').setXFrameOptionsMode(HtmlService.XFrameOptionsMode.DEFAULT);
return out;
} else {
return HtmlService.createHtmlOutput('Invalid URLS');
}
}
I'm struggling for a couple of days. Question is simple, is there a way that can I create a server on Raspberry PI that will return current status of GPIO ports in JSON format?
Example:
Http://192.168.1.109:3000/led
{
"Name": "green led",
"Status": "on"
}
I found Adafruit gpio-stream library useful but don't know how to send data to JSON format.
Thank you
There are a variety of libraries for gpio interaction for node.js. One issue is that you might need to run it as root to have access to gpio, unless you can adjust the read access for those devices. This is supposed to be fixed in the latest version of rasbian though.
I recently built a node.js application that was triggered from a motion sensor, in order to activate the screen (and deactivate it after a period of time). I tried various gpio libraries but the one that I ended up using was "onoff" https://www.npmjs.com/package/onoff mainly because it seemed to use an appropriate way to identify changes on the GPIO pins (using interrupts).
Now, you say that you want to send data, but you don't specify how that is supposed to happen. If we use the example that you want to send data using a POST request via HTTP, and send the JSON as body, that would mean that you would initialize the GPIO pins that you have connected, and then attach event handlers for them (to listen for changes).
Upon a change, you would invoke the http request and serialize the JSON from a javascript object (there are libraries that would take care of this as well). You would need to keep a name reference yourself since you only address the GPIO pins by number.
Example:
var GPIO = require('onoff').Gpio;
var request = require('request');
var x = new GPIO(4, 'in', 'both');
function exit() {
x.unexport();
}
x.watch(function (err, value) {
if (err) {
console.error(err);
return;
}
request({
uri: 'http://example.org/',
method: 'POST',
json: true,
body: { x: value } // This is the actual JSON data that you are sending
}, function () {
// this is the callback from when the request is finished
});
});
process.on('SIGINT', exit);
I'm using the npm modules onoff and request. request is used for simplifying the JSON serialization over a http request.
As you can see, I only set up one GPIO here. If you need to track multiple, you must make sure to initialize them all, distinguish them with some sort of name and also remember to unexport them in the exit callback. Not sure what happens if you don't do it, but you might lock it for other processes.
Thank You, this was very helpful. I did not express myself well, sorry for that. I don't want to send data (for now) i just want to enter web address like 192.168.1.109/led and receive json response. This is what I manage to do for now. I don't know if this is the right way. PLS can you review this or suggest better method..
var http = require('http');
var url = require('url');
var Gpio = require('onoff').Gpio;
var led = new Gpio(23, 'out');
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/html'});
var command = url.parse(req.url).pathname.slice(1);
switch(command) {
case "on":
//led.writeSync(1);
var x = led.readSync();
res.write(JSON.stringify({ msgId: x }));
//res.end("It's ON");
res.end();
break;
case "off":
led.writeSync(0);
res.end("It's OFF");
break;
default:
res.end('Hello? yes, this is pi!');
}
}).listen(8080);
Is it possible to show the thumbnails for an embedded YouTube playlist that is subscribing to a feed over https instead of http?
I'm working on an area of a site that is https and retrieving these thumbnails over http is causing a security warning pop-up, which I am trying to get rid of.
The thumbnails are being requested as part of the YouTube API so unable to directly request them over https:
// loop through JSON and assign values to My obj
for (var i = 0; i < len; i ++ ) {
TheVideo.My[i] = {};
ja = jsonArticles[0][i];
TheVideo.My[i].img = ja.media$group.media$thumbnail[0].url;
}
What i'm directly looking for is to call the thumbnail from https instead of http when .url is called in the last line above.
EDIT:
I've found a work around to the problem, but I've edited my original post to clarify a bit what the problem was.
The answer is a bit of an add-on but it does the job. I appended a replace method to request it from https instead of http:
// loop through JSON and assign values to My obj
for (var i = 0; i < len; i ++ ) {
TheVideo.My[i] = {};
ja = jsonArticles[0][i];
TheVideo.My[i].img = ja.media$group.media$thumbnail[0].url.replace('http://','https://');
}
Nice to know that this is possible in these situations.
Hope that this can potentially answer questions for someone else.
Gurus of SO
I have posted a web app to the iOS Home Screen & want to not have to login each time the app opens up. So I am trying to push the cookie into LocalStorage.
I am using the following code to try to store my cookies in LocalStorage for a mobile web app (code copied from iphone web app ruby gem). But somehow its not working. Any suggestions?
Thank you.
<script type="text/javascript">
(function(){
var RESEND_REQUEST = {{RESEND}};
function isFullScreen(){
return navigator.userAgent.match(/WebKit.*Mobile/) &&
!navigator.userAgent.match(/Safari/);
}
if(isFullScreen()){
if(document.cookie == ''){
var storedValues = localStorage.getItem('__cookie__');
if(storedValues){
var values = storedValues.split(';');
for(var i=0; i < values.length; i++)
document.cookie = values[i];
}
document.cookie = '_cookieset_=1';
if(RESEND_REQUEST){
window.location.reload();
}
}
var lastCookie = null;
setInterval(function(){
if(lastCookie != ''+document.cookie){
lastCookie = ''+document.cookie;
localStorage.setItem('__cookie__', ''+document.cookie);
}
},1000);
}
})()
There are couple thing that does fit in the above code
1. if(document.cookie == '')
The above statement not always suppose return true even when you are opening your web_app from iOS Home Screen for the first time i.e the document.cookie does contain some value (junk though but still) even opening from Home screen(atleast what I found). I urge you to prompt the same with alert
Something like alert(document.cookie) before running into the above mentionif clause
If yes(document.cookie does contain some value) then I guess you need to fix the above if clause something like this
> if(!document.cookie.match(/_session_id/) ) {
> // Rest of the code goes here
> }
if your using ActiveRecord::Base.session_store
or
> if (!document.cookie.match(/{{YOUR SESSION KEY}}/) {
> // Rest of the code goes here
> }
your Session Key if using Cookie Store "the following key can be found my looking at the config/initializer/session_store.rb file
2. As notice the below code
localStorage.setItem('__cookie__', ''+document.cookie)
does make sense when reading though it but there is twist to it
one would except the document.cookie to contain cookie for the application maintained
and stored by the browser but as I notice that document.cookie does not turn out to be same
e.g browser stored the following cookie for my application
"__cookieset=1;KBD=0en-3;_session_id=896c455928f3dd9e7bb0b660efb7063c"
but when inspected the document.cookie I found it to be contain
"__cookieset=1;KBD=0en-3;"
Notice that document.cookie doesnot contain "_session_id=896c455928f3dd9e7bb0b660efb7063c"
Which is must as It used by various authorization gem(devise or authlogic) to determine whether the current user has a valid session ?
so I request you store the cookie from the request object obtain from Rack::Request.new(env)
into the localStorage
3. The middleware placement make sure your placing middleware at right place.
If your using ActiveRecord::Base.session_store I guess the patch code of the same gem can be found here solve your purpose
I have this code which works with no errors when sending "simple text" through Ajax post.
But when I want to send "html" to the server, my code will fail sometimes.
var message = tinyMCE.activeEditor.getContent();
if(message.length > 0)
{
message = Base64.encode(message);
tinyMCE.activeEditor.setContent('');
var parameters = 'message=' + message;
var url = "ChatResponse.aspx";
ajaxPost.open("POST", url, true);
ajaxPost.setRequestHeader("Content-Type", "application/x-www-form-urlencoded");
ajaxPost.setRequestHeader("Content-Length", parameters.length);
ajaxPost.setRequestHeader("Connection", "close");
ajaxPost.send(parameters);
}
Will not fail for
asdasd<strong>asdasd</strong> [base64:]
YXNkYXNkPHN0cm9uZz5hc2Rhc2Q8L3N0cm9uZz4=
but will fail for
a<strong>a</strong> [base64:]
YTxzdHJvbmc+YTwvc3Ryb25nPg==
What is wrong?
It is not being encoded before being sent. I believe TinyMCE has a method that allows you to get the encoded content. You may also want to try taking the content from the textarea using the value property.