I have uploaded an excel and need to read it and get the count of the number of rows in it.
I have tried the below codes but it doesn't seen to work out.
I have been getting so many errors like,"_fs.readFileSync is not a function" "f.slice is not a function" and currently I am stuck at "input.replace is not a function". Please let me know where am I going wrong or what actually needs to be done
Component.ts
<pre>
proceedUpload(){
if(this.file != null) {
var workbook = XLSX.read(this.file);
var firstSheet = workbook.SheetNames[0];
var excelRows = XLSX.utils.sheet_to_json(workbook.Sheets[firstSheet]);
var number = excelRows.length();
if( number > 0 )
{
this.uploadMessage = true;
}
else
{
this.uploadMessage = false;
}
let formData: FormData = new FormData();
formData.append('Files', this.file);
this.http.post("Url", formData).map(res => res.json())
.catch(error => Observable.throw(error))
.subscribe(
data => console.log('success'),
error => console.log(error)
)
}
else
{
alert("Please upload a file!")
}
</pre>
Component.html
<div *ngIf="uploadMessage">
<p>{{number}} uploaded!!</p>
</div>
const table = XLSX.readFile('mytable.xlsx');
const sheet = table.Sheets[table.SheetNames[0]];
var range = XLSX.utils.decode_range(sheet['!ref']);
console.log(range.e.r);
Source
let excelRowsObjArr = XLSX.utils.sheet_to_row_object_array(workbook.Sheets[firstSheet]);
On this one you can do:
excelRowsObjArr.length
For more info you can go here: https://www.aspsnippets.com/Articles/Read-Parse-Excel-File-XLS-and-XLSX-using-JavaScript.aspx
Related
I'm attempting to gather all of the posts submitted to a particular subreddit using the code found here: https://www.labnol.org/internet/web-scraping-reddit/28369/ However the execution limit is reached well before this completes.
I am looking for a way to extend the run time of the script, with it ideally not needing my intervention at all once I click run.
const getThumbnailLink_ = url => {
if (!/^http/.test(url)) return '';
return `=IMAGE("${url}")`;
};
const getHyperlink_ = (url, text) => {
if (!/^http/.test(url)) return '';
return `=HYPERLINK("${url}", "${text}")`;
};
const writeDataToSheets_ = data => {
const values = data.map(r => [
new Date(r.created_utc * 1000),
r.title,
getThumbnailLink_(r.thumbnail),
getHyperlink_(r.url, 'Link'),
getHyperlink_(r.full_link, 'Comments')
]);
const sheet = SpreadsheetApp.getActiveSheet();
sheet.getRange(sheet.getLastRow() + 1, 1, values.length, values[0].length).setValues(values);
SpreadsheetApp.flush();
};
const isRateLimited_ = () => {
const response = UrlFetchApp.fetch('https://api.pushshift.io/meta');
const { server_ratelimit_per_minute: limit } = JSON.parse(response);
return limit < 1;
};
const getAPIEndpoint_ = (subreddit, before = '') => {
const fields = ['title', 'created_utc', 'url', 'thumbnail', 'full_link'];
const size = 10000;
const base = 'https://api.pushshift.io/reddit/search/submission';
const params = { subreddit, size, fields: fields.join(',') };
if (before) params.before = before;
const query = Object.keys(params)
.map(key => `${key}=${params[key]}`)
.join('&');
return `${base}?${query}`;
};
const scrapeReddit = (subreddit = 'AskMen') => {
let before = '';
do {
const apiUrl = getAPIEndpoint_(subreddit, before);
const response = UrlFetchApp.fetch(apiUrl);
const { data } = JSON.parse(response);
const { length } = data;
before = length > 0 ? String(data[length - 1].created_utc) : '';
if (length > 0) {
writeDataToSheets_(data);
}
} while (before !== '' && !isRateLimited_());
};
Generally it's a better practice to optimize your script to not reach the execution time defined by your quota. So in your case, one solution is to reduce the batch size per execution. In the reference you linked the code fetches 1000 posts per batch, your code fetches 10000.
Try with smaller values to see if the script execution time does not exceed quota anymore.
const getAPIEndpoint_ = (subreddit, before = '') => {
const fields = ['title', 'created_utc', 'url', 'thumbnail', 'full_link'];
const size = 1000;
const base = 'https://api.pushshift.io/reddit/search/submission';
const params = { subreddit, size, fields: fields.join(',') };
if (before) params.before = before;
const query = Object.keys(params)
.map(key => `${key}=${params[key]}`)
.join('&');
return `${base}?${query}`;
};
But if you have a business need to exceed your quota, you can upgrade to to either Google Workspace Basic, Business or Enterprise - depending on how much you need to increase your quota and how much you are willing to pay.
See here for more information about different accounts and pricing.
I’ve searched a script that automates google form from my question banks (g-sheets).
All of my questions are multiple choice. I already added the setPoints() and setRequired(), however, for the answer key, I can’t find a way on how to script the value of my Answer’s column into TRUE when it meets the criteria. Below are the codes:
function getSpreadsheetData(sheetName) {
// This function gives you an array of objects modeling a worksheet's tabular data, where the first items — column headers — become the property names.
var arrayOfArrays = SpreadsheetApp.openById("GoogleSheetID").getSheetByName(sheetName || 'question').getDataRange().getValues();
console.log("Spreadsheet: " + arrayOfArrays[0]);
console.log("Spreadsheet: " + arrayOfArrays[1]);
var headers = arrayOfArrays.shift();
return arrayOfArrays.map(function (row) {
return row.reduce(function (memo, value, index) {
if (value) {
memo[headers[index]] = value;
}
return memo;
}, {});
});
}
function onOpen(e){
var form = FormApp.openById("GoogleFormID");
form.setTitle('DO ANY TITLE YOU WANT HERE');
form.setDescription('This is an example of Form generation');
getSpreadsheetData().forEach(function (row) {
var capitalizedName = row.Number.charAt(0).toUpperCase() + row.Number.slice(1);
console.log("Spreadsheet: " + capitalizedName);
form.addPageBreakItem()
.setTitle(capitalizedName);
var item = form.addMultipleChoiceItem();
item.setTitle(row.Questions)
.setPoints(1)
.setRequired(true)
.setChoices([
item.createChoice(row.Option1),
item.createChoice(row.Option2),
item.createChoice(row.Option3),
item.createChoice(row.Option4)
]);
});
}
function onSubmit(e) {
var form = FormApp.getActiveForm();
var items = form.getItems();
while(items.length > 0){
form.deleteItem(items.pop());
}
}
I’ve also search that to make choices the right answer, you just put TRUE or something like this:
item.createChoice(row.Option1, true),
item.createChoice(row.Option2, false),
item.createChoice(row.Option3, false),
item.createChoice(row.Option4, false)
However, this will only set Option1 as always the right answer. I want that TRUE or FALSE will be automatically place when found the right answer as stated in column G of my google sheets. Attach is my sheet:
GoogleSheet
This took me a while and I experience a weird issue, which is: After the Questions are set and the form saved, refreshing the page or trying to answer deletes all the options from one of the options (not always the same option).
I still have no clue why does this happen and I will investigate it later, but my code can give you an idea.
I changed your code a bit and added a new function which returns true or false for each option depending on the value of the G column:
form.addPageBreakItem()
.setTitle(capitalizedName);
var item = form.addMultipleChoiceItem();
item.setTitle(row.Questions)
.setPoints(1)
.setRequired(true)
.setChoices([
item.createChoice(row.Option1, checktrue(row, row.Option1)),
item.createChoice(row.Option2, checktrue(row, row.Option2)),
item.createChoice(row.Option3, checktrue(row, row.Option3)),
item.createChoice(row.Option4, checktrue(row, row.Option4))
]);
});
}
function checktrue(row, option){
var numRow = 2;
var sprsheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
var colA = sprsheet.getRange("A2:A5").getValues();
for (var i = 0; i < colA.length; i++){
if (colA[i] == row.Number){
numRow += i;
}
}
var answer = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet().getRange("G"+numRow).getValue();
if (option == answer){
return true
} else {
return false
}
}
I cannot seem to get this script to work for 'statistics':
function searchByKeyword2() {
var results = YouTube.Search.list('statistics', {q: 'dogs', maxResults: 5, });
for(var i in results.items) {
var item = results.items[i];
Logger.log(item);
}
}
I can use 'id', 'snippet', or 'id, snippet', but I cannot get it to work with 'statistics'. I've been looking an answer for hours, but I haven't found anything. Any clues?
Per the API documentation, YouTube.Search includes results for Videos, Channels, and Playlists. Not all of these resources have statistics nodes, and thus the YouTube.Search endpoint does not allow querying for the statistics node - only id and snippet.
For collections which track statistics, like Videos, you query them directly to access statistics. Since Videos.list does not search, you need to first search and then provide the relevant video IDs. Note that you can change the search order (and many other search properties) - full details available in the API reference - but the default sort is 'relevance'.
As an example:
function getVideoStatistics(videoIds) {
const options = {
id: videoIds.join(","),
fields: "nextPageToken,items(id,statistics)"
};
const results = [];
do {
var search = YouTube.Videos.list('statistics', options);
if (search.items && search.items.length)
Array.prototype.push.apply(results, search.items);
options.pageToken = search.nextPageToken;
} while (options.pageToken);
return results;
}
function getVideosFromQuery(query, maxResults) {
const options = {
q: query,
maxResults: maxResults,
type: 'video',
fields: "nextPageToken,pageInfo/totalResults,items(id/videoId,snippet(title,channelTitle))"
};
const results = [];
do {
var search = YouTube.Search.list('snippet', options);
if (search.items && search.items.length)
Array.prototype.push.apply(results, search.items);
options.pageToken = search.nextPageToken;
} while (options.pageToken && results.length < search.pageInfo.totalResults && results.length < maxResults);
return results;
}
function foo() {
var someQuery = "something";
var searchResults = getVideosFromQuery(someQuery, 50);
var ids = searchResults.map(function (videoSearchResult) {
return videoSearchResult.id.videoId;
});
var stats = getVideoStatistics(ids);
console.log({message:"query video statistics", searchResults: searchResults, statistics: stats});
}
Is there is a fast way to programmatically export all responses from a Google Form to a csv? Something like "Export responses to csv" invoked via Scripts.
Right now I'm doing it in a rock art way:
Iterate over the forms I want to export (~75)
Open each form var form = FormApp.openById(formId);
Get responses: var formReponses = form.getResponses(); (from 0 to 700 responses each form)
Iterate over responses and get item responses: var preguntes = formReponses[r].getItemResponses();
For each itemResponse, convert it to csv/json
Export responses to a drive file
This is extremly slow and additionally it hangs over and over, so I had to export responses in chunks of 50 responses and save them in Drive separated files. On next execution (after letting servers to cool down for a while), I'm executing the script again, skipping the number of responses found on the chunk file.
Additionally I'm not sure that Google keeps the responses order when doing form.getResponses(); (actually I've found that if the form has been modified, the order is not the same)
Is there a better way to do it?
Whith the help of #JackBrown I've managed to write a Chrome extension to download responses (maybe soon in github). This will wait for each download in the formIds object until finished and then prompt for the next one:
'use strict';
function startDownload() {
const formIds = {
'Downloads-subfolder-here': {
'Download-filename-here': '1-cx-aSAMrTK0IHsQkE... {form-id here}',
'Another-filename-here': '...-dnqdpnEso {form-id here}',
// ...
},
'Another-subfolder-here': {
'Download-filename-here': '1-cx-aSAMrTK0IHsQkE... {form-id here}',
'Another-filename-here': '...-dnqdpnEso {form-id here}',
// ...
},
};
const destFolders = Object.keys(formIds);
const downloads = [];
for (let t = 0, tl = destFolders.length; t < tl; t += 1) {
const destFolder = destFolders[t];
const forms = Object.keys(formIds[destFolder]);
for (let f = 0, fl = forms.length; f < fl; f += 1) {
const formName = forms[f];
downloads.push({
destFolder,
formName,
url: `https://docs.google.com/forms/d/${formIds[destFolder][formName]}/downloadresponses?tz_offset=-18000000`,
filename: `myfolder/${destFolder}/${formName.replace(/\//g, '_')}.csv`,
});
}
}
const event = new Event('finishedDownload');
const eventInterrupt = new Event('interruptedDownload');
let currId;
chrome.downloads.onChanged.addListener((downloadDelta) => {
if (downloadDelta.id === currId) {
if (downloadDelta.state && downloadDelta.state.current === 'complete') {
document.dispatchEvent(event);
} else if (downloadDelta.state && downloadDelta.state.current === 'interrupted') {
console.log(downloadDelta);
document.dispatchEvent(eventInterrupt);
}
}
});
downloads.reduce((promise, actual) => {
return promise.then((last) => (last ? new Promise((resolve) => {
const { url, filename, destFolder, formName } = actual;
function listener() {
document.removeEventListener('finishedDownload', listener);
document.removeEventListener('interruptedDownload', listener);
resolve(true);
};
function interrupt() {
document.removeEventListener('finishedDownload', listener);
document.removeEventListener('interruptedDownload', listener);
resolve(false);
}
console.log(`Processant ${destFolder}, ${formName}: ${url}`);
document.addEventListener('finishedDownload', listener);
document.addEventListener('interruptedDownload', interrupt);
chrome.downloads.download({ url, filename }, (downloadId) => {
currId = downloadId;
if (!downloadId) {
console.log();
console.log('Error downloading...');
console.log(runtime.lastError);
resolve();
}
});
}) : Promise.resolve(false)));
}, Promise.resolve(true));
}
chrome.browserAction.onClicked.addListener((/*tab*/) => startDownload());
var sender_username = req.session.user_id;
var recipient_username = req.body.recipient_username;
var content = req.body.content;
var sql = ' SELECT sender_username, recipient_username, COUNT(recipient_username) as count FROM message WHERE sender_username = "'+sender_username+'" AND recipient_username = "'+recipient_username+'" GROUP BY sender_username LIMIT 1 ';
var message_no = 0;
var data;
connection.query(sql, function(err, result) {
if (err) {
res.send(err);
}
else {
data = result;
// res.send(data); < - this works
// res.send(result); <- this works
// res.send(result.count); <- undefined
}
});
res.send(data); // undefined (can't seem to save to variable after connection.query())
The res.send(result); seems to work. It gives:
[{"sender_username":"sender","recipient_username":"recipient","count":2}]
I am just trying to get the value for count and save that to a variable, but things like result.count are returning undefined for some reason.
It's because the JSON is an array, so you should access it like the following
res.send(result[0].count);