I try to import some data from a secure site with sigh into a spreadsheet in Google Drive.
I can retrieve information like the title or some menus, but the private content returns #N/A value.
I have this chunk of html, and i need to get the Issues:
<div class="gh-box-section">
<ul class="gh-ul-aa">
<li>
<div class="gh-group">
<em>Bug:</em> 37
</div>
</li>
<li class="gh-summary">
<div class="gh-group">
<em>Total Issues:</em> 37
</div>
</li>
</ul>
I guess I can not get the sensitive information for some permission or login, I try to retrieve the information opening my session in one Chrome tab and make the import in another tab, but it did not work.
so, my question: is there a way to start a session for another site from Google Drive and import sensitive data from the html code?
Related
I'm trying to import some energy data into a Google Sheet, but can't seem to get the value to appear since it is just text in the HTML line, and shows up as --,--- in the output. I have tried adding /text() to the XPath but that is still not working either.
Website with data to import:
http://www.caiso.com/TodaysOutlook/Pages/default.aspx
Formula in Google Sheets:
=IMPORTXML("http://www.caiso.com/TodaysOutlook/Pages/default.aspx","//div[#class='overview-large-number']")
Also tried the follwoing Xpath with the same --,--- being displayed.
/html/body/div[2]/div[3]/div[4]/div/div/div[2]/div/div[1]/div[1]/div[1]
/html/body/div[2]/div[3]/div[4]/div/div/div[2]/div/div[1]/div[1]/div[1]/text()
Screen Shot of Webpage:
Caiso Webage with Data
HTML Element with Data:
<div class="overview-large-number">41,946 <span>MW</span></div>
Screen Shote of Google Sheets current Output:
Google Sheets Screenshot
Thank you in advanced for any insight.
The data is populated after the page loads, so you can't use ImportXML(). The data is requested via HTTP Get on the URL http://www.caiso.com/outlook/SP/stats.txt and is returned in JSON format.
Get the ImportJSON script from here and add it to your sheet (Tools > Script Editor).
This should be enough to get you started:
=Transpose(ArrayFormula(HLOOKUP(
{"AvailableCapacity","CurrentSystemDemand","todayForecastPeakDemand","histDemand","TodaysPeakDemand","tomorrowsForecastPeakDemand"},
ImportJSON("http://www.caiso.com/outlook/SP/stats.txt"),2,0)))
Note that the Historical Peak value is already a string, so you may have to do some extra work to get the number out.
Recently I noticed that after searching for some term in Google and browsing the search result, the relevant web page showed my relevant search text as highlighted.
So, I noticed what Google has done is, as sending extra parameters by appending to the URL as #:~:text=this%20text%20will%20be%20highlighted.
As an example if I go to https://en.wikipedia.org/wiki/Stack_Overflow, it shows as follows;
But when I append #:~:text=Overflow to the URL and visit https://en.wikipedia.org/wiki/Stack_Overflow#:~:text=Overflow, then it shows as follows (note that word 'Overflow' is highlighted);
What I'm wondering is how this behavior is achieved?
What are such other types of such parameters that we could use?
Edit:
My browser is: Microsoft Edge, Version 83.0.478.45 (Official build) (64-bit)
Below are some more common and less advanced URL parameters but, I'll post them here anyways:
There are of course Anchor links that can be appended to the URL
<!-- https://Example.com/#top -->
<body>
<a name="top"> </a>
<a href="#top">
Go To Top Of Page
</a>
</body>
And URL query parameters that can be picked up by the server in a variety of ways, here is a python/Django example
# https://Example.com/?variable=someValue
import requests
def page_view(request):
variableValue = request.GET.get('variable', None)
I am trying to embed, into Google Classic Sites, a project from Google Scripts that contains a clickable image pointing to a separate page from within this same Google Classic Site (there are reasons I am not using the Sites editor directly and inserting as an image with a link to the page).
The problem is when I click on the image/anchor, I get the dreaded "sites.google.com refused to connect." with no errors logged in Cloud Platform but console.log error is this: Refused to display 'https://sites.google.com/a/DOMAIN/SITE_NAME/PAGE_NAME/' in a frame because it set 'X-Frame-Options' to 'sameorigin'.
How I have embedded into Classic Sites from Scripts:
from Scripts => Publish/Deploy as web app with settings:
Execute the app as: tried both me and as user
Who has access to the app: Anyone within DOMAIN
from Scripts => Test web app for your latest code results: same as when embedded in Classic Sites
from Classic Sites => Edit Page / Insert / Apps Script, then pasted URL of published Scripts project.
Here is the code from Google Apps Script I am embedding:
Code.gs file
function doGet() {
return HtmlService.createTemplateFromFile('Buttons').evaluate()
.setTitle('Client Home Page');
}
Buttons.html file
<!DOCTYPE html>
<html>
<head>
<?!= HtmlService.createHtmlOutputFromFile('stylesheet').getContent(); ?>
</head>
<body>
<a href="https://sites.google.com/a/DOMAIN/SITES_NAME/PAGE_NAME/" imageanchor="1" target="" >
<img src="https://sites.google.com/a/DOMAIN/IMAGE_LINK" border="0" alt="Image" >
</a>
</body>
</html>
What I have also tried:
Embedding the Script as an iFrame gadget. Results: doesn't show UI at all.
Adding an onclick event to fetch URL instead (see code changes below). Results: embedded area where buttons display turns blank on click with no errors logged in Cloud Platform or console.
fetchUrl attempt: changes to Code.gs file
function doGet() {
return HtmlService.createTemplateFromFile('Buttons').evaluate()
.setTitle('Home');
}
function setUrl(url) {
var response = UrlFetchApp.fetch(url);
return response.getContentText();
}
fetchUrl attempt: changes to anchor tag in Buttons.html file
<a href="" onclick="google.script.run.setUrl('https://sites.google.com/a/DOMAIN/SITES_NAME/PAGE_NAME/')" imageanchor="1" target="" >
<img src="https://sites.google.com/a/DOMAIN/IMAGE_LINK" border="0" alt="Image" >
</a>
Any help or insight would be greatly appreciated!
Leora
Figured it out! I was missing the <base> tag in my HTML file (<base target="_top">).
While this is automatically supplied in any new HTML file you create within Google Apps Scripts, I had copied and pasted from code written in VS Code, therefore omitting it.
As soon as I added it back in everything works as expected. Going to leave this here in case anyone else runs into this problem.
Many thanks to anyone that attempted to look into this...
Leora
I am using UWP store app using typescript and winjs. As per requirement we need to load remote web page in our app and that page required some form parameter which are not possible to set with query string as size limitation. so we decided to use IFrame which allow to receive form post data. for this I have add the remote web page URI to the ApplicationContentUriRules section of the package manifest.
but getting me below error while performing post form action.
APPHOST9625: Unable to navigate to: 'https://www.mytestweb.com/'. An
iframe attempted to navigate to a URI that is not included in the
ApplicationContentUriRules for this app. Use a x-ms-webview element to
view the URI instead, or add the URI to the ApplicationContentUriRules
section of the package manifest so that the iframe can navigate to it.
(In Visual Studio, add this URI to the Content URIs tab of the
Manifest Designer.)
Is anything I am missing or wrong? I need help to resolve this.
below is my sample code,
<form id="my-form" action="https://www.mytestweb.com/operations" method='post' target="my-iframe">
<div class="col">
<div class="row">
<input type="hidden" name="token" value="largeTokenString" />
</div>
<div class="row">
<input type="hidden" name="transaction" value="serializedTransactionData" />
</div>
<div class="buttonContainer">
<button type="submit" data-bind="click: postFromData">Post</button>
</div>
</div>
</form>
<iframe id="my-iframe" name="my-iframe" src="about:blank"></iframe>
When I submit on from post button remote URL loads with form data, without from data my remote URL is not loaded.
I am able to resolve my issue. One of my team member question me, how you know your app refers updated app manifest. I don't know where I find this.
So I have uninstall my app from computer clear app memory from app data using app name search and delete. Create new app package using msbuild command and install new app, now my app able to load remote URL successfully in IFrame.
I've created a file in Google Apps Script as follows
DocsList.createFile(
"test.csv",
"Row1Col1,Row1Col2 \r\n Row2Col1,RowCol2 \r\n Row3Col1,Row3Col2");
How to download it programmatically (via a popup Download dialog for example) ?
Try using ContentService, you can trigger a download and even set the file name via the TextOutput.downloadAsFile method. You can also set the content mime type of the returned data, though only to a predefined enum constant. See their examples.
To download a file with apps script you need to either publish the app, and use the doGet() method (otherwise the "downloadFileAs" won't work)
OR...
create the file in Drive, and provide a link in a dialog ui.
here are two options for links, one requires users to be logged in the other doesn't taken from here
var dat=getFileDataFromAnotherFunction();
var file = DriveApp.createFile('name.csv',dat);
var t = HtmlService.createTemplateFromFile('DownloadDialog');
t.url1 = getDlUrl(file);
t.url2 = file.getDownloadUrl().replace('&gd=true','')
rt = t.evaluate().setSandboxMode(HtmlService.SandboxMode.IFRAME);
uia.showModalDialog(rt,'Download the csv')
where "DownloadDialog.html"
is the following file (also in the script editor )
<div>
<a target="_blank" href="<?= url1 ?>"> l1 </a>
<br>
<hr>
<a target="_blank" href="<?= url2 ?>"> link 2 </a>
</div>
If you want to show a download dialog at client end then following code works. It is an example for CSV file.
return ContentService.createTextOutput("bbb,aaa,ccc").downloadAsFile("MyData.csv")
.setMimeType(ContentService.MimeType.CSV);