I'm working on a project where I'm looking to store raw svg data inside my mongodb. Right now, it appears a bit goofy because I need to escape the svg string like so:
{ "_id" : ObjectId("4c61e60d4d02da615f175b6e"), "name" : "Triangle", "svg-data" : "<?xml version=\"1.0\" encoding=\"utf-8\"?> <!-- Generator: Adobe Illustrator 14.0.0, SVG Export Plug-In . SVG Version: 6.00 Build 43363) --> <!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\"> <svg version=\"1.1\" id=\"Layer_1\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" x=\"0px\" y=\"0px\" width=\"63.781px\" height=\"85.042px\" ... bunch of data elided here ... </svg>" }
This is somewhat acceptable but then I may need to de-escape (is that a word?) or decode on the client side once the json is passed back down from the server.
I'm curious if anyone else is doing this (I couldn't find any examples on google or stack overflow) and/or if there is any advice on the best way to do this (or a better way to do it).
Personally I haven't stored files in MongoDB yet, but according to mongodb.org it's quite efficient to store files in the database. Storing files in the database gives you the following advantages:
Because the files are chunked, you can fetch only part of a file. Useful for objects such as video, not so much for SVG.
All of your files are replicated together with your database, so you don't have to set up a separate replication system just for your files.
Now, the thing that seems to be holding you back is escaping the data. However, you only have to escape the SVG data if you manually type statements into the MongoDB shell. If you're using a driver to interact with the database, the driver will take care of escaping special characters in the data. When you read out the data in your code, the driver will also unescape the data for you.
An example to demonstrate, in the MongoDB shell:
var myObject = { "myKey": "Data with \"special\" characters." }
print(myObject.myKey)
In the first line I have to escape the quotes, because I manually type the statement. The MongoDB drivers will do this for you if you interact with the database from code.
The print() statement automatically unescapes the data, so the output will be:
Data with "special" characters.
Database drivers will also unescape strings automatically. If you send a JSON object to the client's browser and read out the values with JavaScript, the browser will also unescape the data.
When you manually prepare the data for insertion, you will have to escape special characters, there's no way around it. But as soon as you interact with the database from code, the database drivers will take care of this and you won't have to worry about it.
Related
I am loading data from a mongodb collection to a mysql table through Kettle transformation.
First I extract them using MongodbInput and then I use json input step.
But since json input step has very low performance, I wanted to replace it with a
javacript script.
I am a beginner in Javascript and even though i tried somethings, the kettle javascript script is not recognizing any keywords.
can anyone give me sample code to convert Json data to different columns using javascript?
To solve your problem you need to see three aspects:
Reading from MongoDB
Reading from JSON
Reading from (probably) String
Reading from MongoDB Except if you changed the interface, MongoDB returns not JSON but BSON files (~binary JSON). You need to see the MongoDB documentation about reading and writing BSON: probably something like BSON.to() and BSON.from() but I don't know it by heart.
Reading from JSON Once you have your BSON in JSON format, you can read it using JSON.stringify() which returns a String.
Reading from (probably) String If you want to use the capabilities of JSON (why else would you use JSON?), you also want to use JSON.parse() which returns a JSON object.
My experience is that to send a JSON object from one step to the other, using a String is not a bad idea, i.e. at the end of a JavaScript step, you write your JSON object to a String and at the beginning of the next JavaScript step (can be further down the stream) you parse it back to JSON to work with it.
I hope this answers your question.
PS: writing JavaScript steps requires you to learn JavaScript. You don't have to be a master, but the basics are required. There is no way around it.
you could use the json input step to get the values of this json and put in common rows
So there's a small debate by my team, and I'm sure this is answered in many places but I couldn't find any definitive answers.
Right now we have a server that tosses up JSON data (think REST, sorta). The client is a complete JavaScript client that uses $.ajax to grab the data and render it appropriately.
The client is using UnderscoreJS templates to render data within the HTML:
<%- something %>
So if the server sends down a JSON block (non-html encoded):
{
"username": "Joe's Crab & Cookies"
}
Should the server be HTML or JavaScript encoding this value? Or should that still be left up to the client?
What if a bit of data from the server needs to be an attribute of an element:
<li data-item-id="<%= userId %>">something</li>
I realize that I shouldn't need to encode anything that's generated by the server, it's all data that is entered by the user. So imagine the "userId" above being set by a user, not generated.
So if we encode on the server and on the client we see on the rendered page:
Joe's Crab & Cookies
If you're sending json data somewhere, the only encoding that should be done to it is json encoding. You don't necessarily know if the values are going to end up in sql, javascript, xml, html attributes, a winforms app, etc.
Now, on the other hand, if some of your json values were to contain html, that html value should be encoded, ready-to-display html. It depends on context.
First of all, you should be escaping the value, if it can be set by a user.
You should use as much escaping and validation as possible -- both on the input fields, when capturing the data on the server, when inputting into DB and, finally, when rendering it back.
Among the mild consequences of not escaping data would be that it can crack your HTML when you're outputting to data-item-id.
I have a solution where I need to be able to log multiple JSON Objects to a file. Essentially doing one log file per day. What is the easiest way to write (and later read) these from a single file?
How does MongoDB handle this with BSON? What does it use as a separator between "records"?
Does Protocol Buffers, BSON, MessagePack, etc... offer compression and the record concept? Compression would be a nice benefit.
With protocol buffers you could define the message as follows:
Message JSONObject {
required string JSON = 1;
}
Message DailyJSONLog {
repeated JSONObject JSON = 1;
}
This way you would just read the file from memory and deserialize it. Its essentially the same way for serializing them as well. Once you have the file (serialized DailyJSONLog) on disk, you can easily just append serialized JSONObjects to the end of that file (since the DailyJSONLog message is very simply a repeated field).
The only issue with this is if you have a LOT of messages each day or if you want to start at a certain location during the day (you're not able to easily get to the middle (or arbitrary) of the repeated list).
I've gotten around this by taking a JSONObject, serializing it and then base64 encoding it. I'd store these to a file separating by a new line. This allows you to very easily see how many records are in each file, gain access to any arbitrary JSON object within the file and to trivially keep expanding the file (you can expand the above 'repeated' message as well pretty trivially but it is a one way easy operation...)
Compression is a different topic. Protocol Buffers will not compress strings. If you were to define a pb message to match your JSON message, then you will get the benefit of having pb possibly 'compress' any integers into their [varint][1] encoded format. You will get 'less' compression if you try above base64 encoding route as well.
I have RaphaelJS based canvas, where user can do some work. I need to do 2 things:
Save users work into database
Rastersize SVG (I use Apache Batik for that task)
For 1 https://github.com/jspies/raphael.serialize dumps Raphael's canvas to json, then jQuery.getJSON() to send it to the backend, where I need to convert it back to SVG to feed into Batik.
Does this flow seem reasonable? SVG -> JSON -> SVG conversion seems a bit overhead, is there a better way to do this?
Backend runs python/django. I use standard json package for JSON -> SVG, but sometimes it fails with syntax errors in the incoming json (mainly in font style properties). Did anyone face these issues?
What's the best way to store this data in the database? Just as a string?
Raphael.Export saves elements to SVG in any browser that Raphaël supports:
https://github.com/ElbertF/Raphael.Export
I've used it to save Raphaël drawings as PNGs server-side using Batik.
If you can get a handle on the root <svg> element from Raphael then you can convert this to the raw XML source (on the client) and just send that:
var svgAsXML = (new XMLSerializer).serializeToString(svg);
I'm writting an application in Sencha-Touch 1.1 and I want to add localization to strings.
I've thought (but not implemented) of two ways.
Create seperate JSON files for each language (en-US.json, el-GR.json
etc) and use a proxy and a store to load the data, changing each
time the proxy url (json file destination). The problem is that I don't know how to extract the data from the store afterwards.
Create a global JSON object which then I inflate (parse the json file
and turn it into an object). The problem here is that I cannot seem to find a way to parse a JSON file without using a reader/proxy combo.
Is there any optimal solution for localizing strings in sencha-touch?
A common approach is to extract all of your strings into class level properties and have separate locale files which override these classes and their string properties.
For example, if I had a panel like this:
MyApp.MyPanel = Ext.extend(Ext.Panel, {
myPanelContent: 'This is your text to be translated...',
initComponent: function(){
Ext.apply(this, {
html: this.myPanelContent
});
MyApp.MyPanel.superclass.initComponent.call(this)
}
});
And then you would have a locale file along the lines of:
Ext.override(MyApp.MyPanel, {
myPanelContent: 'This is a translation of my text to another language'
});
You can then either load the correct locale file when your app is loaded (after the views themselves have been loaded) or dynamically load the locale files during runtime (this would require logic to update each of the views with the current value).
Hope this makes sense, give me a shout if it doesn't!
Stuart