Import data through VBA from webpage - json

The following link has a button as "Export to Excel". When clicked, it downloads an Excel sheet with details for the particular stock.
I want to get this downloaded excel file through Excel VBA, where I have a set of links it loops through them download books from each link, copy certain data and paste it. While writing the code I realised that for downloading from web, the link should end with xlsx or csv or something related to excel.
Sub transfercsv()
sLink = "http://www.screener.in/company/?q=500095" 'I can loop and attach links to this..
sfile = "downloaded.xls" 'I can get this rectified as well...
ssheet = "Data sheet"
'Further code I can build up
....
End sub
Final question: "How can I ask Excel VBA to download the workbook behind the Export to Excel link."
or
"How can I find the xlsx link behind the Export to Excel button."
P.S:- I have tagged as JSON as well, becasuse while researching I found the the export link has something to do with JSON..

This is Pratyush from Screener.in.
The above won't work because of these technical reasons:
The export function uses POST request.
The page is behind a user login.
The post request has CSRF protection to prevent automation.
This has been done because generating customized XLSX files is heavy and server dynamic process, and being on a shared host we have limited resources.
A better alternative can be to use Google Docs worksheets. They have a wonderful API integrated with Google Finance which can provide much of the financial data.

Related

How to access and extract data in App Script that is not bound to the code

I am working on a script that is standalone. It collects its baseline data from a google sheet bound to a form. In the form submission, excel documents are uploaded. While I can access the link for the excel document(google downloads the excel into your drive upon submission and inserts the drive link into the google response sheet), I am having difficulty looking into and extracting anything (exact cell values, ranges, indexes, etc) from the excel doc.
I have tried using various functions from the sheets and spreadsheets classes and continue to get errors. Some suggestions I've found say the excel document needs to be converted into a google sheet before the app script can access it, or that app script won't allow you to work with such a document if it is not bound to your script (since that allows you to activate it)(I can't bind the excel doc because it changes upon every new submission of the form).
Has anyone ever compiled a similar code and figured out how to access an unbound, non-google doc?
Let me know if you need error codes or script snippets. I just wasn't sure if this was a syntax problem or a google suite trick spot that needed extra code that I probably haven't found cause I'm new to this platform.
From the question
Has anyone ever compiled a similar code and figured out how to access an unbound, non-google doc?
Scripts in Google Apps Script aren't "compiled" by the script writer in the sense that it's done when developing something in other platforms.
To get data from a file hosted in Google Drive, first the script should get that file by using the Drive Service (Class DriveApp) or the Drive Advanced Service.
The next part depends pretty much on the Excel file format. If it's an xlsx file, then usually the most convenient is to convert the file into an Google spreadsheet as this will make possible to use the Spreadsheet Service (Class SpreadsheetApp) to read the data from it.
If you don't want to convert it to a Google spreadsheet file, or it can't be converted the the "basic" means because it's using an incompatible format (like a xls file format), then you will need to use an library or an external service to parse the Excel file content.
function getalldataonsheet() {
const ss=SpreadsheetApp.openById('ssid');//you provide id
const sh=ss.getSheetByName('sheetname');//you provide sheet namme
const rg=sh.getDataRange();
const vs=rg.getValues();//2d array
return vs;//this return 2d array
}
There are restrictions for passing parameters in client to server communication

Download a file from a link on another site?

I am attempting to use Google Apps Script to download an excel file that is linked to on a CRM. That is, instead of each day having a person log in to the CRM, download new data as an excel file, and copy and paste the new data to our existing Google Sheet, I would like to have my script do so.
Is it possible to do this? I have been looking at UrlFetchApp, however I'm having trouble because there is no direct link to the file I need (only a link to generate and download the file).

Google forms script to save responses into a csv (not excel) by default and email it as attachment

I currently have a google form which appends responses to an Excel format file. I have also attached a script to it which parses the response and emails the results...
Building on that, how would I do the following additional steps:
Keep the current behavior which appends responses to the default EXCEL spreadsheet format file.
Convert the default EXCEL spreadsheet format file to a file called allresonses.txt (CSV format)
Saves the current responses as mytest.txt (CSV format) and have this emailed as an attachment. (Currently I am able to only add them to an email using my script)
Any help would be appreciated
Thank you
The solution is to use scripts , I check the google forms script examples.
Quick walk through:
1. Just create a form in google forms and add your questions.
2. Create a script on the add script page (of course google for the correct script or check the developer forum examples)
3. While the default is a spreadsheet format you can select one of the supported types (again I check the google developer examples,they're somewhat dated but do still work.)
All sorted now.
Thank you

Google Drive Web API does not return Open Office export link for new Spreadsheet version

this morning I noticed that the current way we used to export our Google Spreadsheets as Open Office (ods) documents does not work anymore with the new spreadsheet version. We were requesting the file's metadata via the Google Drive API and used the returned exportLinks. For old version spreadsheets the result was something like this:
exportLinks: {
'application/pdf': 'https://docs.google.com/feeds/download/spreadsheets/Export?key=someid&exportFormat=pdf',
'application/x-vnd.oasis.opendocument.spreadsheet': 'https://docs.google.com/feeds/download/spreadsheets/Export?key=someid&exportFormat=ods',
'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet': 'https://docs.google.com/feeds/download/spreadsheets/Export?key=someid&exportFormat=xlsx'
}
So we got a PDF, XLSX and ODS link. But in the new version we get a CSV instead of an ODS link:
exportLinks: {
'text/csv': 'https://docs.google.com/spreadsheets/export?id=someid&exportFormat=csv',
'application/pdf': 'https://docs.google.com/spreadsheets/export?id=someid&exportFormat=pdf',
'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet': 'https://docs.google.com/spreadsheets/export?id=someid&exportFormat=xlsx'
}
According to the documentation the export links for a spreadsheet should be Microsoft Excel (xlsx), Open Office sheet (ods) and PDF. I'm currently not sure if the documentation is not up to date or I'm doing something wrong with the API. My current workaround was to hardcode the link to the ODS export. The download is still working, the download link is just missing from exportLinks object.
P.S.: To make it more clear. Everything is still working fine with spreadsheets that are not yet updated to the new spreadsheet version.
I solved this problem with
download_url = drive_file[u'exportLinks'][u'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet']
# xlsx shold be changed to ods
download_url = download_url.replace('xlsx','ods')
but I don't know how much time it will work

Replicate with Google Script the behaviour of "File" > "Download as..." in Google Spreadsheet

I'm looking for a way to either:
a) replace "File" > "Download as..." > "Microsoft Excel (.xlsx)" in Google Spreadsheet in such a way that when I click on "Microsoft Excel (.xlsx)" it asks me for the name of the file that I want to save (i.e.: like a typical "Save as..." menu entry).
b) if it's not possible to replace the standard menu entry, I would like at least to add another menu entry in another menu which behaves like described above.
Even though I haven't been able to write a script that lets me replace an existing menu entry, I've managed to write a script that adds an additional menu.
Is it possible to accomplish "a"?
Another problem now is that I can't seem to find a way to replicate the behaviour of "File" > "Download as..." > "Microsoft Excel (.xlsx)".
I've found other posts in which it's described how to write Google Scripts in order to do something similar (e.g.: create a backup of a spreadsheet in Google Drive in a specific format, or send it via GMail to a specific address).
Actually, after reading "Can I download file from URL link generated by google apps script" I'm not even sure it's possible.
Note that you cannot download per se, since you have no access to your
PC's resources (e.g. file system) from apps-script. The file is still
in "the cloud"... in this case, it's been copied from the web site it
was on, into Google Drive. If you're running the Drive app, though,
the file will now sync to your PC.
Is there any way to do that?
Thanks in advance.
The only way I achieved download of a file from the spreadsheet is by writing completely separate script which accessed my original spreadsheet by document id, generated a file and downloaded it like this:
function doGet(){
var outputDocument = DocumentApp.create('Some csv');
var content = getCsv();
var textContent = ContentService.createTextOutput(content);
textContent.setMimeType(ContentService.MimeType.CSV);
textContent.downloadAsFile("4NocniMaraton.csv");
return textContent;
}
Then I had to publish the App and get a link for it. That link I placed in one of the sheets in the original spreadsheet, which I used as a data download link. Other then this I didn't find a way to work the security issues around.