Dart / flutter: download or read the contents of a Google Drive file - google-drive-api

I have a public (anyone with the link can view) file on my Google Drive and I want to use the content of it in my Android app.
From what I could gather so far, I need the fileID, the OAuth token and the client ID - these I already got. But I can't figure out what is the exact methodology of authorising the app or fetching the file.
EDIT:
Simply reading it using file.readAsLines didn't work:
final file = new File(dogListTxt);
Future<List<String>> dogLinks = file.readAsLines();
return dogLinks;
The dogLinks variable isn't filled with any data, but I get no error messages.
The other method I tried was following this example but this is a web based application with explicit authorization request (and for some reason I was never able to import the dart:html library).
The best solution would be if it could be done seamlessly, as I would store the content in a List at the application launch, and re-read on manual refresh button press.
I found several old solutions here, but the methods described in those doesn't seem to work anymore (from 4-5 years ago).
Is there a good step-by-step tutorial about integrating the Drive API in a flutter application written in dart?

I had quite a bit of trouble with this, it seems much harder than it should be. Also this is for TXT files only. You need to use files.export() for other files.
First you need to get a list fo files.
ga.FileList textFileList = await drive.files.list(q: "'root' in parents");
Then you need to get those files based on ID (This is for TXT Files)
ga.Media response = await drive.files.get(filedId, downloadOptions: ga.DownloadOptions.FullMedia);
Next is the messy part, you need to convert your Media object stream into a File and then read the text from it. ( #Google, please make this easier.)
List<int> dataStore = [];
response.stream.listen((data) {
print("DataReceived: ${data.length}");
dataStore.insertAll(dataStore.length, data);
}, onDone: () async {
Directory tempDir = await getTemporaryDirectory(); //Get temp folder using Path Provider
String tempPath = tempDir.path; //Get path to that location
File file = File('$tempPath/test'); //Create a dummy file
await file.writeAsBytes(dataStore); //Write to that file from the datastore you created from the Media stream
String content = file.readAsStringSync(); // Read String from the file
print(content); //Finally you have your text
print("Task Done");
}, onError: (error) {
print("Some Error");
});

There currently is no good step-by-step tutorial, but using https://developers.google.com/drive/api/v3/manage-downloads as a reference guide for what methods to use in Dart/Flutter via https://pub.dev/packages/googleapis: to download or read the contents of a Google Drive file, you should be using googleapis/Drive v3, or specifically, the methods from the FilesResourceApi class.
drive.files.export(), if this is a Google document
/// Exports a Google Doc to the requested MIME type and returns the exported content. Please note that the exported content is limited to 10MB.
drive.files.get(), if this something else, a non-Gdoc file
/// Gets a file's metadata or content by ID.
Simplified example:
var drive = new DriveApi(http_client);
drive.files.get(fileId).then((file) {
// returns file
});
However, what I discovered was that this Dart-GoogleAPIs library seemed to be missing a method equivalent to executeMediaAndDownloadTo(outputStream). In the original Google Drive API v3, this method adds the alt=media URL parameter to the underlying HTTP request. Otherwise, you'll get the error, which is what I saw:
403, message: Export requires alt=media to download the exported
content.
And I wasn't able to find another way to insert that URL parameter into the current request (maybe someone else knows?). So as an alternative, you'll have to resort to implementing your own Dart API to do the same thing, as hinted by what this OP did over here https://github.com/dart-lang/googleapis/issues/78: CustomDriveApi
So you'll either:
do it through Dart with your own HttpClient implementation and try to closely follow the REST flow from Dart-GoogleAPIs, but remembering to include the alt=media
or implement and integrate your own native-Android/iOS code and use the original SDK's convenient executeMediaAndDownloadTo(outputStream)
(note, I didn't test googleapis/Drive v2, but a quick examination of the same methods looks like they are missing the same thing)

I wrote this function to get file content of a file using its file id. This is the simplest method I found to do it.
Future<String> _getFileContent(String fileId) async {
var response = await driveApi.files.get(fileId, downloadOptions: DownloadOptions.fullMedia);
if (response is! Media) throw Exception("invalid response");
return await utf8.decodeStream(response.stream);
}
Example usage:
// save file to app data folder with 150 "hello world"s
var content = utf8.encode("hello world" * 150);
driveApi.files
.create(File(name: fileName, parents: [appDataFolder]),
uploadMedia: Media(Stream.value(content), content.length))
.then((value) {
Log().i("finished uploading file ${value.id}");
var id = value.id;
if (id != null) {
// after successful upload, read the recently uploaded file content
_getFileContent(id).then((value) => Log().i("got content is $value"));
}
});

Related

How would you create a downloadable pdf in a client side app?

One of our requirements for an admin tool is to create a form that can be filled and translated to a downloadable pdf file. (A terms and condition with blank input fields to be exact).
I did some googling and tried creating a form in html and css and converted it into a canvas using the html2canvas package. Then I used the jspdf package to convert it into a pdf file. The problem is that I cannot get it to fit and resize accordingly to an a4 format with correct margins. I'm sure I can get to a somewhat working solution if I spend some time on it.
However, my real question is how would you guys solution this? Is there a 3rd party app/service that does this exact thing? Or would you do all this in the server side? Our current app is using angular 7 with firebase as our backend.
Cheers!
I was able to use the npm package pdfmake to create a dynamic pdf based on user information the user provided while interacting with my form. (I was using React) It opened the pdf in a new tab and the user is able to save the pdf. In another application (still React),
I used the same package to create a receipt so you can customize the size of the "page". We created the pdf and used the getBase64() method and sent the pdf as an email attachement.
My service function:
getEvidenceFile(id: number, getFileContent: boolean) {
return this.http.get(environment.baseUrl + ‘upload’ + ‘/’ + id , {responseType: ‘blob’ as ‘json’})
.map(res => res);
}
My component function called from the selected item of a FileDownload…
FileDownload(event: any) {
// const blob = await this.callService.getEvidenceFile(event.target.value, true);
// const url = window.URL.createObjectURL(blob);
this.callService.getEvidenceFile(event.target.value, true).subscribe(data => {
var binaryData = [];
binaryData.push(data);
var downloadLink = document.createElement(‘a’);
downloadLink.href = window.URL.createObjectURL(new Blob(binaryData));
document.body.appendChild(downloadLink);
downloadLink.click();
});
}

Can't import geojson value as string in google maps with firebase web

So, I set up my firebase to communicate with my web app which uses google maps api and my goal is this: When a user draws a shape on the map(polygon, linestring), I want to send the geoJson value of it to the firebase(currently sending it as a String), and then retrieve it back so it appears on the map for everyone(since it's getting synced from the firebase database). My problem is that when I try to retrieve the geoJson data back and add it on google maps, at the line map.data.addGeoJson(geoJsonString);(geoJsonString = geoJson value that is stored in firebase) I get an error saying:
Uncaught Jb {message: "not a Feature or FeatureCollection", name: "InvalidValueError", stack: "Error↵ at new Jb (https://maps.googleapis.com/m…tatic.com/firebasejs/4.13.0/firebase.js:1:278304)"}
For some reason google maps api doesnt accept the geoJson value even though console.log(geoJsonString); returns a valid geoJson value (checked at http://geojsonlint.com/)
Now the strange part is that if I try to import the same geoJson value manually(storing the geoJson value in a var and then map.data.addGeoJson(geoJsonString);) it works just fine.
This function syncs firebase with the web app
function gotData(data){
paths = data.val();
if(paths == null){
console.log("firebase null");
alert('Database is empty! Try adding some paths.');
}
else{
var keys = Object.keys(paths);
for(var i = 0; i < keys.length; i++){
var k = keys[i];
var geoJsonString = paths[k].geoJsonString;
console.log(geoJsonString);
map.data.addGeoJson(geoJsonString);
}
}
}
This function updates and pushes data in firebase
function updateData(){
data = {
geoJsonString: geoJsonOutput.value
}
ref = database.ref('firebasePaths');
ref.push(data);
}
In this function(which is used to store geoJson values locally in a file), I call updateData function), after a new path is drawn on the map
// Refresh different components from other components.
function refreshGeoJsonFromData() {
map.data.toGeoJson(function(geoJson) {
geoJsonOutput.value = JSON.stringify(geoJson);
updateData();
refreshDownloadLinkFromGeoJson();
});
}
Example of my firebase that contains 2 random geoJson
I can't trace where the problem is. Any ideas?
Update: I managed to fix this issue by parsing the string with JSON.parse("retrieved string from firebase"), saving it to a variable and then adding it to the map with map.data.addgeoJson(parsed variable).
We still have not faced that issue, however, we are aware of it.
Our intended solution is to use GeoFire: An open-source library for the Firebase Realtime Database that adds support for geospatial querying.
You can find the library description in here:
https://firebase.google.com/docs/libraries/
For the Web supported library:
https://github.com/firebase/geofire-js

How to parse json newline delimited in Angular 2

I am writing an Angular 2 app (built with angular cli), and trying to use AWS Polly text-to-speech API.
According to the API you can request audio output as well as "Speech Marks" which can describe word timing, visemes, etc. The audio is delivered as "mp3" format, and the speech marks as "application/x-json-stream", which I understand as a "new line" delimited JSON. It cannot be parsed with JSON.parse() due to the new lines. I have yet been unable to read/parse this data. I have looked at several libs that are for "json streaming" but they are all built for node.js and won't work with Angular 2. My code is as follows...
onClick() {
AWS.config.region = 'us-west-2';
AWS.config.accessKeyId = 'xxxxx';
AWS.config.secretAccessKey = 'yyyyy';
let polly = new AWS.Polly();
var params = {
OutputFormat: 'json',
Text: 'Hello world',
VoiceId: 'Joanna',
SpeechMarkTypes:['viseme']
};
polly.synthesizeSpeech(params, (err, data) => {
if (err) {
console.log(err, err.stack);
} else {
var uInt8Array = new Uint8Array(data.AudioStream);
var arrayBuffer = uInt8Array.buffer;
var blob = new Blob([arrayBuffer]);
var url = URL.createObjectURL(blob);
this.audio.src = url;
this.audio.play(); // works fine
// speech marks info displays "application/x-json-stream"
console.log(data.ContentType);
}
});
Strangely enough Chrome browser knows how to read this data and displays it in the response.
Any help would be greatly appreciated.
I had the same problem. I saved the file so I could then read it line by line, accessing the JSON objects when I need to highlight words being read. Mind you this is probably not the most effective way, but an easy way to move on and get working on the fun stuff.
I am trying out different ways to work with Polly, will update answer if I find a better way
You can do it with:
https://www.npmjs.com/package/ndjson-parse
That worked for me.
But I can't play audio, I tried your code it says
DOMException: Failed to load because no supported source was found.

How write and immediately read a file nodeJS

I have to obtain a json that is incrusted inside a script tag in certain page... so I can't use regular scraping techniques, like cheerio.
Easy way out, write the file (download the page) to the server and then read it using string manipulation to extract the json (there are several) work on them and save to my db hapily.
the thing is that I'm too new to nodeJS, and can't get the code to work, I think that I'm trying to read the file before it is fully written, and if read it time before obtain [Object Object]...
Here's what I have so far...
var http = require('http');
var fs = require('fs');
var request = require('request');
var localFile = 'tmp/scraped_site_.html';
var url = "siteToBeScraped.com/?searchTerm=foobar"
// writing
var file = fs.createWriteStream(localFile);
var request = http.get(url, function(response) {
response.pipe(file);
});
//reading
var readedInfo = fs.readFileSync(localFile, function (err, content) {
callback(url, localFile);
console.log("READING: " + localFile);
console.log(err);
});
So first of all I think you should understand what went wrong.
The http request operation is asynchronous. This means that the callback code in http.get() will run sometime in the future, but the fs.readFileSync, due to its synchronous nature will execute and complete even before the http request will actually be sent to the background thread that will execute it, since they are both invoked in what is commonly known as the (same) tick. Also fs.readFileSync returns a value and does not use a callback.
Even if you replace fs.readFileSync with fs.readFile instead the code still might not work properly since the readFile operation might execute before the http response is fully read from the socket and written to the disk.
I strongly suggest reading: stackoverflow question and/or Understanding the node.js event loop
The correct place to invoke the file read is when the response stream has finished writing to the file, which would look something like this:
var request = http.get(url, function(response) {
response.pipe(file);
file.once('finish', function () {
fs.readFile(localFile, /* fill encoding here */, function(err, data) {
// do something with the data if there is no error
});
});
});
Of course this is a very raw and not recommended way to write asynchronous code but that is another discussion altogether.
Having said that, if you download a file, write it to the disk and then read it all back again to the memory for manipulation, you might as well forgo the file part and just read the response into a string right away. Your code will then look something like so (this can be implemented in several ways):
var request = http.get(url, function(response) {
var data = '';
function read() {
var chunk;
while ( chunk = response.read() ) {
data += chunk;
}
}
response.on('readable', read);
response.on('end', function () {
console.log('[%s]', data);
});
});
What you really should do IMO is to create a transform stream that will strip away all the data you need from the response, while not consuming too much memory and yielding this more elegantly looking code:
var request = http.get(url, function(response) {
response.pipe(yourTransformStream).pipe(file)
});
Implementing this transform stream, however, might prove slightly more complex. So if you're a node beginner and you don't plan on downloading big files or lots of small files than maybe loading the whole thing into memory and doing string manipulations on it might be simpler.
For further information about transformation streams:
node.js stream api
this wonderful guide by substack
this post from strongloop
Lastly, see if you can use any of the million node.js crawlers already out there :-) take a look at these search results on npm
According to the http module help 'get' does not return the response body
This is modified from the request example on the same page
What you need to do is process the response with in the callback (function) passed into http.request so it can be called when it is ready (async)
var http = require('http')
var fs = require('fs')
var localFile = 'tmp/scraped_site_.html'
var file = fs.createWriteStream(localFile)
var req = http.request('http://www.google.com.au', function(res) {
res.pipe(file)
res.on('end', function(){
file.end()
fs.readFile(localFile, function(err, buf){
console.log(buf.toString())
})
})
})
req.on('error', function(e) {
console.log('problem with request: ' + e.message)
})
req.end();
EDIT
I updated the example to read the file after it is created. This works by having a callback on the end event of the response which closes the pipe and then it can reopen the file for reading. Alternatively you can use
req.on('data', function(chunk){...})
to process the data as it arrives without putting it into a temporary file
My impression is that you serializing a js object into JSON by reading it from a stream that's downloading a file containing HTML. This is do-able yet hard. Its difficult to know when you're search expression is found because if you parse as the chunks come in then you never know if you received only context and you could never find what you're looking for because it was split into 2 or many parts which were never analyzed as a whole.
You could try something like this:
http.request('u/r/l',function(res){
res.on('data',function(data){
//parse data as it comes in
}
});
This allows you to read data as it comes in. You can handle it to save to disc, db, or even parse it if you accumulated the contents within the script tags into a single string then parsed objects in that.

How to set the modifiedDate on a google drive file in iOS AND prevent duplicate file names

I found the way in iOS (not obvious to me) to insert a file into a google drive folder using the standard queryForFilesInsertWithObject with uploadParameters and setting the folder id in a GTLDriveParentReference object, adding that object to an array, and assigning that array to the uploaded file's "parents" property. What I haven't yet figured out is how to overwrite any existing google drive file with a newer version without creating a duplicate. I would also like to stamp the new uploaded google drive file with the same date-time stamp as my local device file. When I try to use the file/update method and set the file.modifiedDate and query.setModifiedDate=YES, I always get a 400 Bad Request.
I can use patch to change the title just fine, but I can't set the modifiedDate using the api. I was able to change the modifiedDate using Google's Try it! APIs Explorer, so Google's servers are working fine. Here is the code that fails:
GTLDriveFile *file = [GTLDriveFile object];
NSString *fId = insertedFile.identifier;
file.title = #"Tony.jpg";
file.modifiedDate = myNewGoogleDate; // "2012-12-31T08:00:00+00:00"
GTLQueryDrive *qu = [GTLQueryDrive queryForFilesPatchWithObject:file fileId:fId];
qu.setModifiedDate = YES;
[service executeQuery:qu completionHandler:^(GTLServiceTicket *ticket,...
Take out the two lines with modifiedDate and the google-drive file title changes just fine. I traced into the api call to executeQuery and found that the following results as "DataToPost".
{"method":"drive.files.patch","id":"gtl_7","jsonrpc":"2.0",
"params": {"setModifiedDate":true,"resource":
{"modifiedDate":"2012-12-31T08:00:00+00:00",
"title":"Tony.jpg"},"fileId":"0BxRlXPR_hfR9OEJSdDdyQjkyaE0"},"apiVersion":"v2"}
Does anyone see "Bad Request" in this post? The error returned from the executeQuery is:
Error Domain=com.google.GTLJSONRPCErrorDomain Code=400 "The operation couldn’t be
completed. (Bad Request)" UserInfo=0x80724b0 {error=Bad Request,
GTLStructuredError=GTLErrorObject 0x806a7a0: {message:"Bad Request" code:400 data:[1]},
NSLocalizedFailureReason=(Bad Request)}
Does anyone have some code to set the modifiedDate that works?
You want to use queryForFilesUpdateWithObject instead to update the file using the original's fileId to ensure that the correct file is updated.
+ (id)queryForFilesUpdateWithObject:(GTLDriveFile *)object
fileId:(NSString *)fileId
uploadParameters:(GTLUploadParameters *)uploadParametersOrNil {
The duplicate file name is avoided if you supply the same pre-existing identifier (as probably explained in the answer above). The modifiedDate can be set using the GTLDateTime construction. Both solutions are shown below:
NSString *fileID = gF.identifier;
gF.modifiedDate = [GTLDateTime dateTimeWithDate:myNSDate timeZone:[NSTimeZone localTimeZone]];
if (fileID==0) query = [GTLQueryDrive queryForFilesInsertWithObject:gF uploadParameters:upParams];
else query = [GTLQueryDrive queryForFilesUpdateWithObject:gF fileId:fileID uploadParameters:upParams];
query.setModifiedDate = YES;
Below code is working with GoogleAPIClient v3 and i couldn't see any bad request.
driveFile = [GTLDriveFile object];
driveFile.modifiedTime = [GTLDateTime dateTimeWithDate:[NSDate date] timeZone:[NSTimeZone localTimeZone]];
output:
(lldb) po driveFile.modifiedTime
GTLDateTime 0x157a06470: {2016-08-10T13:44:33.609+05:30}