I am writing an Angular 2 app (built with angular cli), and trying to use AWS Polly text-to-speech API.
According to the API you can request audio output as well as "Speech Marks" which can describe word timing, visemes, etc. The audio is delivered as "mp3" format, and the speech marks as "application/x-json-stream", which I understand as a "new line" delimited JSON. It cannot be parsed with JSON.parse() due to the new lines. I have yet been unable to read/parse this data. I have looked at several libs that are for "json streaming" but they are all built for node.js and won't work with Angular 2. My code is as follows...
onClick() {
AWS.config.region = 'us-west-2';
AWS.config.accessKeyId = 'xxxxx';
AWS.config.secretAccessKey = 'yyyyy';
let polly = new AWS.Polly();
var params = {
OutputFormat: 'json',
Text: 'Hello world',
VoiceId: 'Joanna',
SpeechMarkTypes:['viseme']
};
polly.synthesizeSpeech(params, (err, data) => {
if (err) {
console.log(err, err.stack);
} else {
var uInt8Array = new Uint8Array(data.AudioStream);
var arrayBuffer = uInt8Array.buffer;
var blob = new Blob([arrayBuffer]);
var url = URL.createObjectURL(blob);
this.audio.src = url;
this.audio.play(); // works fine
// speech marks info displays "application/x-json-stream"
console.log(data.ContentType);
}
});
Strangely enough Chrome browser knows how to read this data and displays it in the response.
Any help would be greatly appreciated.
I had the same problem. I saved the file so I could then read it line by line, accessing the JSON objects when I need to highlight words being read. Mind you this is probably not the most effective way, but an easy way to move on and get working on the fun stuff.
I am trying out different ways to work with Polly, will update answer if I find a better way
You can do it with:
https://www.npmjs.com/package/ndjson-parse
That worked for me.
But I can't play audio, I tried your code it says
DOMException: Failed to load because no supported source was found.
Related
So im trying to automate a task at work, and im wondering if theres anyway to send data from the console of one webpage to the console of another web page.
The task i am trying to automate consists of a website that has a prefilled form. I need to get elements from this form, and then copy them into another totally different website. Ive already written a script that pulls the data i need from the form and displays it in the console. Now I need to find a way to send the data (which is simply variables) to the other page’s console. Is this possible?
Keep in mind this is in a work computer, not allowed to download anything on it.
Are you an admin of the webpages and are these pages from the same site? if the answer is yes, i would recommend you use localStorage for saving and retrieving the data then display it to the console.
If it's not your website and you want it to work anyway just create a simple browser extension.
Here are some links to help you get started with extensions
MDN doc
Chrome doc
The idea is for you to target webpage A collect the data and post it to Github
Then target webpage B to read data from your github gist and you dispaly it in the console.
Cheers, i hope it was helpfull
Which server side language are you using ?
Usually for these, you could just have a form which is posting data to another website's form.
Look at this php example :
https://www.ostraining.com/blog/coding/retrieve-html-form-data-with-php/
Correct me If I did not understand your question correctly.
//Store the logs in following way
console.stdlog = console.log.bind(console);
console.logs = [];
console.log = function(){
console.logs.push(Array.from(arguments));
console.stdlog.apply(console, arguments);
}
//copying the logs into a json file
(function(console){
console.save = function(data, filename){
if(!data) {
console.error('Console.save: No data')
return;
}
if(!filename) filename = 'console.json'
if(typeof data === "object"){
data = JSON.stringify(data, undefined, 4)
}
var blob = new Blob([data], {type: 'text/json'}),
e = document.createEvent('MouseEvents'),
a = document.createElement('a')
a.download = filename
a.href = window.URL.createObjectURL(blob)
a.dataset.downloadurl = ['text/json', a.download, a.href].join(':')
e.initMouseEvent('click', true, false, window, 0, 0, 0, 0, 0, false, false, false, false, 0, null)
a.dispatchEvent(e)
}
})(console)
console.save(console.logs) //prints the logs in console.json file
// from the console.json file, you can use log information from another page
//Store the logs in following way
console.stdlog = console.log.bind(console);
console.logs = [];
console.log = function(){
console.logs.push(Array.from(arguments));
console.stdlog.apply(console, arguments);
}
localStorage.setItem('Logs', console.logs);
localStorage.getItem('Logs'); // from any browser
I have a public (anyone with the link can view) file on my Google Drive and I want to use the content of it in my Android app.
From what I could gather so far, I need the fileID, the OAuth token and the client ID - these I already got. But I can't figure out what is the exact methodology of authorising the app or fetching the file.
EDIT:
Simply reading it using file.readAsLines didn't work:
final file = new File(dogListTxt);
Future<List<String>> dogLinks = file.readAsLines();
return dogLinks;
The dogLinks variable isn't filled with any data, but I get no error messages.
The other method I tried was following this example but this is a web based application with explicit authorization request (and for some reason I was never able to import the dart:html library).
The best solution would be if it could be done seamlessly, as I would store the content in a List at the application launch, and re-read on manual refresh button press.
I found several old solutions here, but the methods described in those doesn't seem to work anymore (from 4-5 years ago).
Is there a good step-by-step tutorial about integrating the Drive API in a flutter application written in dart?
I had quite a bit of trouble with this, it seems much harder than it should be. Also this is for TXT files only. You need to use files.export() for other files.
First you need to get a list fo files.
ga.FileList textFileList = await drive.files.list(q: "'root' in parents");
Then you need to get those files based on ID (This is for TXT Files)
ga.Media response = await drive.files.get(filedId, downloadOptions: ga.DownloadOptions.FullMedia);
Next is the messy part, you need to convert your Media object stream into a File and then read the text from it. ( #Google, please make this easier.)
List<int> dataStore = [];
response.stream.listen((data) {
print("DataReceived: ${data.length}");
dataStore.insertAll(dataStore.length, data);
}, onDone: () async {
Directory tempDir = await getTemporaryDirectory(); //Get temp folder using Path Provider
String tempPath = tempDir.path; //Get path to that location
File file = File('$tempPath/test'); //Create a dummy file
await file.writeAsBytes(dataStore); //Write to that file from the datastore you created from the Media stream
String content = file.readAsStringSync(); // Read String from the file
print(content); //Finally you have your text
print("Task Done");
}, onError: (error) {
print("Some Error");
});
There currently is no good step-by-step tutorial, but using https://developers.google.com/drive/api/v3/manage-downloads as a reference guide for what methods to use in Dart/Flutter via https://pub.dev/packages/googleapis: to download or read the contents of a Google Drive file, you should be using googleapis/Drive v3, or specifically, the methods from the FilesResourceApi class.
drive.files.export(), if this is a Google document
/// Exports a Google Doc to the requested MIME type and returns the exported content. Please note that the exported content is limited to 10MB.
drive.files.get(), if this something else, a non-Gdoc file
/// Gets a file's metadata or content by ID.
Simplified example:
var drive = new DriveApi(http_client);
drive.files.get(fileId).then((file) {
// returns file
});
However, what I discovered was that this Dart-GoogleAPIs library seemed to be missing a method equivalent to executeMediaAndDownloadTo(outputStream). In the original Google Drive API v3, this method adds the alt=media URL parameter to the underlying HTTP request. Otherwise, you'll get the error, which is what I saw:
403, message: Export requires alt=media to download the exported
content.
And I wasn't able to find another way to insert that URL parameter into the current request (maybe someone else knows?). So as an alternative, you'll have to resort to implementing your own Dart API to do the same thing, as hinted by what this OP did over here https://github.com/dart-lang/googleapis/issues/78: CustomDriveApi
So you'll either:
do it through Dart with your own HttpClient implementation and try to closely follow the REST flow from Dart-GoogleAPIs, but remembering to include the alt=media
or implement and integrate your own native-Android/iOS code and use the original SDK's convenient executeMediaAndDownloadTo(outputStream)
(note, I didn't test googleapis/Drive v2, but a quick examination of the same methods looks like they are missing the same thing)
I wrote this function to get file content of a file using its file id. This is the simplest method I found to do it.
Future<String> _getFileContent(String fileId) async {
var response = await driveApi.files.get(fileId, downloadOptions: DownloadOptions.fullMedia);
if (response is! Media) throw Exception("invalid response");
return await utf8.decodeStream(response.stream);
}
Example usage:
// save file to app data folder with 150 "hello world"s
var content = utf8.encode("hello world" * 150);
driveApi.files
.create(File(name: fileName, parents: [appDataFolder]),
uploadMedia: Media(Stream.value(content), content.length))
.then((value) {
Log().i("finished uploading file ${value.id}");
var id = value.id;
if (id != null) {
// after successful upload, read the recently uploaded file content
_getFileContent(id).then((value) => Log().i("got content is $value"));
}
});
I want to send streaming data (as sequences of ArrayBuffer) from a Chrome extension to a Chrome App, since Chrome message API (includes chrome.runtime.sendMessage, postMessage...) does not support ArrayBuffer and JS arrays have poor performance, I have to try other methods. Eventually, I found WebRTC over RTCDataChannel might a good solution in my case.
I have succeeded to send string over a RTCDataChannel, but when I tried to send ArrayBuffer I got:
code: 19
message: "Failed to execute 'send' on 'RTCDataChannel': Could not send data"
name: "NetworkError"
It seems that it's not a bandwidths limits problem since it failed even though I sent one byte of data. Here is my code:
pc = new RTCPeerConnection(configuration, { optional: [ { RtpDataChannels: true } ]});
//...
var dataChannel = m.pc.createDataChannel("mydata", {reliable: true});
//...
var ab = new ArrayBuffer(8);
dataChannel.send(ab);
Tested on OSX 10.10.1, Chrome M40 (Stnble), M42(Canary); and on Chromebook M40.
I have filed a bug for WebRTC here.
I modified my code, now everything worked amazing:
removed RtpDataChannels option when creating RTCPeerConnection.(YES, remove RtpDataChannels option if you want data channel, what a magic world!)
on receiver side: no need createDataChannel, instead, handle onmessage, onxxx by using event.channle from pc.ondatachannel callback:
pc.ondatachannel function(event)
var receiveChannel = event.channel;
receiveChannel.onmessage = function(event){
console.log("Got Data Channel Message:", event.data);
};
};
I have to obtain a json that is incrusted inside a script tag in certain page... so I can't use regular scraping techniques, like cheerio.
Easy way out, write the file (download the page) to the server and then read it using string manipulation to extract the json (there are several) work on them and save to my db hapily.
the thing is that I'm too new to nodeJS, and can't get the code to work, I think that I'm trying to read the file before it is fully written, and if read it time before obtain [Object Object]...
Here's what I have so far...
var http = require('http');
var fs = require('fs');
var request = require('request');
var localFile = 'tmp/scraped_site_.html';
var url = "siteToBeScraped.com/?searchTerm=foobar"
// writing
var file = fs.createWriteStream(localFile);
var request = http.get(url, function(response) {
response.pipe(file);
});
//reading
var readedInfo = fs.readFileSync(localFile, function (err, content) {
callback(url, localFile);
console.log("READING: " + localFile);
console.log(err);
});
So first of all I think you should understand what went wrong.
The http request operation is asynchronous. This means that the callback code in http.get() will run sometime in the future, but the fs.readFileSync, due to its synchronous nature will execute and complete even before the http request will actually be sent to the background thread that will execute it, since they are both invoked in what is commonly known as the (same) tick. Also fs.readFileSync returns a value and does not use a callback.
Even if you replace fs.readFileSync with fs.readFile instead the code still might not work properly since the readFile operation might execute before the http response is fully read from the socket and written to the disk.
I strongly suggest reading: stackoverflow question and/or Understanding the node.js event loop
The correct place to invoke the file read is when the response stream has finished writing to the file, which would look something like this:
var request = http.get(url, function(response) {
response.pipe(file);
file.once('finish', function () {
fs.readFile(localFile, /* fill encoding here */, function(err, data) {
// do something with the data if there is no error
});
});
});
Of course this is a very raw and not recommended way to write asynchronous code but that is another discussion altogether.
Having said that, if you download a file, write it to the disk and then read it all back again to the memory for manipulation, you might as well forgo the file part and just read the response into a string right away. Your code will then look something like so (this can be implemented in several ways):
var request = http.get(url, function(response) {
var data = '';
function read() {
var chunk;
while ( chunk = response.read() ) {
data += chunk;
}
}
response.on('readable', read);
response.on('end', function () {
console.log('[%s]', data);
});
});
What you really should do IMO is to create a transform stream that will strip away all the data you need from the response, while not consuming too much memory and yielding this more elegantly looking code:
var request = http.get(url, function(response) {
response.pipe(yourTransformStream).pipe(file)
});
Implementing this transform stream, however, might prove slightly more complex. So if you're a node beginner and you don't plan on downloading big files or lots of small files than maybe loading the whole thing into memory and doing string manipulations on it might be simpler.
For further information about transformation streams:
node.js stream api
this wonderful guide by substack
this post from strongloop
Lastly, see if you can use any of the million node.js crawlers already out there :-) take a look at these search results on npm
According to the http module help 'get' does not return the response body
This is modified from the request example on the same page
What you need to do is process the response with in the callback (function) passed into http.request so it can be called when it is ready (async)
var http = require('http')
var fs = require('fs')
var localFile = 'tmp/scraped_site_.html'
var file = fs.createWriteStream(localFile)
var req = http.request('http://www.google.com.au', function(res) {
res.pipe(file)
res.on('end', function(){
file.end()
fs.readFile(localFile, function(err, buf){
console.log(buf.toString())
})
})
})
req.on('error', function(e) {
console.log('problem with request: ' + e.message)
})
req.end();
EDIT
I updated the example to read the file after it is created. This works by having a callback on the end event of the response which closes the pipe and then it can reopen the file for reading. Alternatively you can use
req.on('data', function(chunk){...})
to process the data as it arrives without putting it into a temporary file
My impression is that you serializing a js object into JSON by reading it from a stream that's downloading a file containing HTML. This is do-able yet hard. Its difficult to know when you're search expression is found because if you parse as the chunks come in then you never know if you received only context and you could never find what you're looking for because it was split into 2 or many parts which were never analyzed as a whole.
You could try something like this:
http.request('u/r/l',function(res){
res.on('data',function(data){
//parse data as it comes in
}
});
This allows you to read data as it comes in. You can handle it to save to disc, db, or even parse it if you accumulated the contents within the script tags into a single string then parsed objects in that.
Using mootools and JsonP I get "invalid label" error in Firefox Error console
JsonP seems to work (I get the data correctly)
{"jsondata":[{"title":"title1","link":"http://xxxx.xxx.xxx","thumbsrc":"http://xxxx.xxx.xxx/17_t.jpg" ,"description":".......","pubDate":"2009-03-09 06:26:00",},{"title":"title2","link":"http://xxxx.xxx.xxx","thumbsrc":"http://xxxx.xxx.xxx/16_t.jpg" ,"description":".......","pubDate":"2009-03-09 06:08:09",}]}
but I get the Invalid label error on "jsondata"
the same file works good with request.json
comma removed... nothing
this is the code I'm using
window.addEvent('domready', function() {
var gallery = $('gallery');
new JsonP('http://myjsoncodeurl',{
onComplete: function(jsonObj) {
addImages(jsonObj.jsondata);
}
}).request();
var addImages = function(images) {
images.each(function(image) {
var el = new Element('div', {'class': 'item'});
var name = new Element('h3').inject(el);
var a1 = new Element('a', {'href': image.link,'html': image.title}).inject(name);
var desc = new Element('span', {'html': image.description}).inject(name, 'after');
var a2 = new Element('a', {'href': image.link}).inject(desc,'after');
var img = new Element('img', {'src': image.thumbsrc}).inject(a2);
el.inject(gallery);
});
};
});
it works with normal request.Json, but JSONP that doesn't like my code :(
the same file works good with
request.json
With JSONP, your response should be returning a JavaScript function call (i.e. callback) with the JSON data passed in as the argument. If your response is a plain old JSON text, it won't work in the context of JSONP. You have to tailor your backend to accept a callback argument and call that callback with the JSON data.
You need to put brackets (normal ones, not curly ones) around your object, because sometimes Javascript gets horribly confused and thinks you're doing a label statement, a statement type that I didn't know existed until I Googled this problem.
https://developer.mozilla.org/en/Core_JavaScript_1.5_Guide/Statements#label_Statement
Try passing your object, {"jsondata":[ ... ]} , as ({"jsondata":[ ... ]}) instead. That seems to sort it.
Putting it in here:
http://json.parser.online.fr/
Shows that its valid, but has the extra comma (which will bork IE, although FF should handle it). If removing the comma doesn't fix it, you'll need to post more of your code to help us find the error.
This could be due to the extra commas after the dates