I am trying to use JSZip to unzip a JSON file but due to my lack of understanding how JSZip works I get the response in a format that I do not know how to use.
So far this is my code:
this.rest.getFile(this.stlLocation).subscribe(
data => {
let JSONFIle = new JSZIP();
JSONFIle.file(data.url, data._body, {binary : true, compression : 'DEFLATE'});
console.log(JSONFIle);
},
err => {
this.msgs.push({severity: 'error', summary: 'Error Message', detail: err});
}
);
So I download a file using an angular 2 service and I use an observable to get the response. When the data is received I finally call JSZip and try to unzip the file but the result of the operation is an intricate object with my data scattered all over the place and buried inside several layers. All I want is the unzipped JSON file that I can open and process.
Thank you for your help,
Dino
after a bit of reading I have realized I was going on the wrong path. If you are downloading the file to a browser, you shouldn't have to do anything. Browsers add the Accept-Encoding: 'deflate' header automatically; it is both unnecessary and not good practice to do this at a DOM/JS level. If you are using NGINX the following link may help you out:
NGINX COMPRESSION AND DECOMPRESSION
Related
I posted a question on this yesterday to the GitHub support page and it got flagged as a duplicate - the original answer is here. This issue doesn't automatically fix like it did for that user. Instead, it seems to come and go with no pattern, so I don't have a good way to replicate it. Some songs will work at one point in time, then they won't a couple minutes later.
Error:
[debug] Encodings: locale cp1252, fs utf-8, out UTF-8, pref cp1252
[debug] youtube-dl version 2020.09.20
[debug] Python version 3.7.8 (CPython) - Windows-10-10.0.19041-SP0
[youtube:search] query "iron man 3 song": Downloading page 1
[debug] exe versions: none
[debug] Proxy map: {}
ERROR: query "song name": Failed to parse JSON caused by JSONDecodeError('Expecting value: line 1 column 1 (char 0)')); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
I get the issue when attempting to extract the data from the video. Here is a snippet of the code I am using:
ydlOps = {
'format': 'bestaudio/best',
'outtmpl': './%(title)s.webm',
'noplaylist': True,
'extractaudio': True,
'audioformat': 'webm',
'default_search': 'ytsearch1',
'quite': True,
'verbose':True,
'version': True
}
with youtube_dl.YoutubeDL(ydlOps) as downloader:
songData = downloader.extract_info(url, download=download)
I have changed the options, tried other options said to have worked, and nothing seems to make a difference. Some will work, then not, then they will again.
I think this is a youtube-dl bug. I wrote a parser for youtube searches and it also broke.
When looking previously at the response from youtube, all the JSON data was stored like this:
window["ytInitialData"] = {...}
So you just had to search through the server's response for the string 'window["ytInitialData"]' to find the relevant JSON and extract it. But now, youtube stores the JSON like this inside the html file sent by the server
var ytInitialData = {...}
This needs to be changed on youtube-dl's part when parsing the results.
What's strange is that sometimes youtube uses the previous version and sometimes it uses the current one. I think it's because the change in javascript is progressively being rolled out accross all youtube servers.
Also note that now, the line containing all the JSON ends with '; ' instead of just ';'. This might also require a change from youtube-dl.
You need to submit a pull request to youtube-dl or wait for somebody to fix it.
I am new to Google Cloud Vision API. I am doing OCR on images primarily for bills and receipts.
For a few images it is working fine, but when I try some other images it gives me this error:
Error: { [Error: Request Admission Denied.]
code: 400,
errors:
[ { message: 'Request Admission Denied.',
domain: 'global',
reason: 'badRequest' } ] }
This is my code:
// construct parameters
const req = new vision.Request({
image: new vision.Image('./uploads/reciept.png'),
features: [
new vision.Feature('TEXT_DETECTION', 1)
]
})
vision.annotate(req).then((res) => {
// handling response
//console.log(res.responses[0].textAnnotations);
var desc=res.responses[0].textAnnotations;
var descarr=[];
for (i = 0; i < desc.length; i++) {
descarr.push(desc[i].description);
}
Ran into this problem as well. It was an image size issue. I don't know what the hard limit is. 4MB worked, but 9MB didn't, it's somewhere in between there.
I was able to work around this by saving the image as another format and submitting that instead; there was something "wrong" (or at least, unexpected by Google) with the image file itself. Not sure about image manipulation in the language you're using (js?), but in Python it was as simple as:
from PIL import Image
bad_image = Image.open(open('failure.jpg', 'rb'))
bad_image.save(open('success.png', 'wb'))
The Best Practices doc says that the image file size should not exceed 4 MB. Based on this responses above, this could be the problem.
Interesting. I ran into the same problem today, using Google's Java client. The way I read James' answer, he had a JPEG file that failed, but worked as a PNG file. In my case, I had a PNG file that failed, but worked as a JPEG.
I had concluded it was a size limitation, as I'd expect JPEGs to be typically smaller than PNGs; however, James' experience suggests otherwise.
I couldn't find any relevant documentation in Google's Javadocs. Since the response is a 400 error, perhaps the Java library is not encoding the image buffer correctly.
Since two days I try to load a template in require.js using the text plugin, but I can't manage that.
define(['jquery', 'text!myView.html'], function ($, myView) {...
Produces the error: XMLHttpRequest cannot load
So I put myView.html to a local Webserver and enabled CORS
define(['jquery', 'text!http://localhost/myView.html'], function ($, myView){...
Produces the error: Uncaught SyntaxError: Unexpected token <
Google told me, I should use the requirejs optimizer, which produces a .js out of my html file. This should solve the problem.
To be honest, I really don't undertand what to write in the build.js.
It's so confusing to me. Here is an example build.js: https://github.com/jrburke/r.js/blob/master/build/example.build.js
The only thing I know is that "optimizeAllPluginResources" option has to be true. But what about all the other options?
build.js:
({
paths: {
myView: "myView.html"
},
optimizeAllPluginResources: true,
name: "myView",
out: "myView.js"})
..doens't work
Let me summarize: I only want to optimize a single html file, to use it my Application, nothing else.
Thanks for help.
I have been getting this error FATAL ERROR: JS Allocation failed - process out of memory and I have pinpointed it to be the problem that I am sending really really large json object to res.json (or JSON.stringify)
To give you some context, I am basically sending around 30,000 config files (each config file has around 10,000 lines) as one json object
My question is, is there a way to send such a huge json object or is there a better way to stream it (like using socket.io?)
I am using: node v0.10.33, express#4.10.2
UPDATE: Sample code
var app = express();
app.route('/events')
.get(function(req, res, next) {
var configdata = [{config:<10,000 lines of config>}, ... 10,000 configs]
res.json(configdata); // The out of memory error comes here
})
After a lot of try, I finally decided to go with socket.io to send each config file at a time rather than all config files at once. This solved the problem of out of memory which was crashing my server. thanks for all your help
Try to use streams. What you need is a readable stream that produces data on demand. I'll write simplified code here:
var Readable = require('stream').Readable;
var rs = Readable();
rs._read = function () {
// assuming 10000 lines of config fits in memory
rs.push({config:<10,000 lines of config>);
};
rs.pipe(res);
You can try increasing the memory node has available with the --max_old_space_size flag on the command line.
There may be a more elegant solution. My first reaction was to suggest using res.json() with a Buffer object rather than trying to send the entire object all in one shot, but then I realize that whatever is converting to JSON will probably want to use the entire object all at once anyway. So you will run out of memory even though you are switching to a stream. Or at least that's what I would expect.
I am trying to read a JSON file with Meteor. I've seen various answers on stackoverflow but cannot seem to get them to work. I have tried this one which basically says:
Create a file called private/test.json with the following contents:
[{"id":1,"text":"foo"},{"id":2,"text":"bar"}]
Read the file contents when the server starts (server/start.js):
Meteor.startup(function() {
console.log(JSON.parse(Assets.getText('test.json')));
});
However this seemingly very simple example does not log anything to the console. If I trye to store it in a variable instead on console.logging it and then displaying it client side I get
Uncaught ReferenceError: myjson is not defined
where myjson was the variable I stored it in. I have tried reading the JSON client side
Template.hello.events({
'click input': function () {
myjson = JSON.parse(Assets.getText("myfile.json"));
console.log("myjson")
});
}
Which results in:
Uncaught ReferenceError: Assets is not defined
If have tried all of the options described here: Importing a JSON file in Meteor with more or less the same outcome.
Hope someone can help me out
As per the docs, Assets.getText is only available on the server as it's designed to read data in the private directory, to which clients should not have access (thus the name).
If you want to deliver this information to the client, you have two options:
Use Assets.getText exactly as you have done, but inside a method on the server, and call this method from the client to return the results. This seems like the best option to me as you're rationing access to your data via the method, rather than making it completely public.
Put it in the public folder instead and use something like jQuery.getJSON() to read it. This isn't something I've ever done, so I can't provide any further advice, but it looks pretty straightforward.
The server method is OK, just remove the extra semi-colon(;). You need a little more in the client call. The JSON data comes from the callback.
Use this in your click event:
if (typeof console !== 'undefined'){
console.log("You're calling readit");
Meteor.call('readit',function(err,response){
console.log(response);
});
}
Meteor!