I'm just wondering if d3.text is faster than d3.json?
The reason behind my question is that I'm reading the source code behind cubism.js and I'm just curious to know if it's done with d3.text because it's faster?
Not really.
The reason the graphic metric uses d3.text is because Graphite doesn’t reply with JSON-formatted data; it has its own raw format. Cubism does use d3.json when the server replies with JSON, as for example with cube metrics.
Under the hood, both d3.text and d3.json both use d3.xhr, so they are going to download the file exactly the same way (via asynchronous XMLHttpRequest). Sure, d3.text doesn’t subsequently run the response through JSON.parse, but you still have to parse the reply somehow. And more often than not I would expect the native JSON.parse to be faster, though it would depend on the exact format.
Related
I'm trying to figure out why when using axios (which uses XMLHttpRequest), parsing a large (4-5mb json) takes about 10 times more than when using just fetch and .json() on the result. Even worse, when using XMLHttpRequest the whole UI becomes unresponsive, while using fetch there might be a tiny block when doing the json parsing but the UI is responsive pretty much throughout the download process.
I can't find any documentation about the internals of fetch, but outdated blogs say it just uses XMLHttpRequest internally. If this is true, then both methods should have similar performance.
Note: This difference was seen on both Android and IOS
What I can find is that the JSON parsing on fetch is done on a lower level than what axios does. With axios it happens later on in the request, but on the react-native package, the parsing of the JSON happens straight after getting the response.
The extra layer of returning the data from XMLHttpRequest in string form to axios, who then starts parsing the data is most likely the impact on the performance.
The react-native version of fetch is also just a polyfill, so it's not that. It's the way how fetch parses the data straight from the XMLHttpRequest that is the difference in performance.
From Mozilla API Documentation
...an easy, logical way to fetch resources asynchronously across the network.
This kind of functionality was previously achieved using XMLHttpRequest. Fetch provides a better alternative that can be easily used by other technologies...
Fetch isn't just a wrapper to XMLHttpRequest. This justifies the performance difference between the two alternatives.
I was reading this article about protobuf and I wondered where to use it in the projects. I read some articles that said google created protobuf to replace XML, but as far as I know in 2008 (the first release) JSON was already there.
I searched more and I found an article that the writer suggested to use it instead of JSON, but I still don't get the idea completely.
So where shall I use it? Any special scenario, or like JSON whenever that I want to transport data? Any other scenarios?
It is useful whenever you want to serialize/deserialize your data. Typical situations include sending your data to someone else over the network, storing it to disk or keeping it in context while performing asynchronous processes.
Here is a brief explanation about the main differences between protocol buffer, json and XML: https://stackoverflow.com/a/14029040/6681872
I would love to use protocol buffers, but I am not sure if they fit my use case. Here it is:
I have a Quiz app. This requires a bunch of data, like categories, questions, a list of answers (and which ones are correct). I do not want to be responsible for entering this data - I would prefer to pass it off to a non-programmer to serialize all this data for me, in either XML or JSON. Then my app would just read in the data file.
Does Google's Protocol Buffers fit my use case? Or should I stick to a more traditional format like XML or JSON?
I think not: Protobuf is a binary format. So then you would need to support a text format like XML or JSON and Protobuf.
Also it does not seem you would benefit from Protobufs better berformance at all.
I need to transfer data (objects) between client and server, and Twisted seems a good way to accomplish this. I've been doing a lot searching but still haven't found any example to understand the basic principle. So any simple code would help.
Thanks!
EDIT
Both client and server are written in python
The data may be large, so I need a fast, reliable transmission ( I've taken a look at producers, is that good?)
Flask is great, but I am using another framework, so the whole networking thing relies on Twisted.
It's hard to tell if your question is more about json, python or twisted, but here's an overview, more can follow once the specifics are known. Perhaps you could add some more info to your question so we can offer more assistance :-)
re Json: Json is just a string with a defined structure. If you are working in python and have an object to send as json, then you need to convert the object to a json string by use of
import json
json.dumps(objectName)
If your client is javascript then instead of json.dumps you might use JSON.stringify(objectname).
If you intend to use javascript for clients then some of the frameworks like jQuery make it very easy.
Pythons json.dumps has a lot of optional arguments, most of which you won't need. You can see the options at https://docs.python.org/2/library/json.html
Python is python, I assume you know how to create and populate objects. Will your client be python or javascript or something else? From a javascript client to a python server you would most likely use Ajax to send requests and get responses.
Twisted allows you to easily create a server that will listen on a given port and, when data arrives, an event will occur that supplies the data received. You can then do whatever you need to with the data. Just be careful about doing blocking things like database inserts since the server may miss some data or otherwise misbehave if you interrupt it's event loop. Twisted can be difficult to learn initially, but it is a very powerful and reliable system that is well proven. One alternative to consider, particularly if your clients are not python, is node.js. In my opinion, node is a little bit easier to grasp initially and there are thousands of add-on modules that let you do almost anything you'd want. I use both twisted and node for different things.
Neither node.js nor twisted are software that you can use to just quickly spin up a server or client without some study and experimentation. To use Twisted or Node.js properly confidently, using all their features and goodness, requires a bit of research and work on your part.
There are excellent frameworks like Flask that can be used to build a server that can react to a number of different Ajax calls from a client - you can have a single server be able to respond to several different kinds of requests instead of having a server for each Ajax type.
This is a small library that serializes an object with all its children to JSON and also parses it back to a fully working object:
https://github.com/Toubs/PyJSONSerialization/
I'm designing a distributed application that will consist of a variety of REST services. Lately I've been going back and forth about whether to implement my REST services using the ASP.NET MVC 4 Web API or OData. Web API seems like it will some day be what I need but right now it's only half baked. Specifically, it only has a partial implementation of OData-style URI querying and doesn't do hypermedia out-of-the-box.
So this forces me to take another long hard look at OData. I really like the URI querying capability and structural hypermedia for lazy loading; I think I will use these features a lot in my application. However, the Atom Pub specification appears to be grossly inefficient.
I recently read a post about an efficient format for OData which mentions "dense JSON" but such a thing does not appear to actually exist. Is this true? And even if there's no such thing as dense JSON, regular JSON is still much more efficient than Atom Pub, correct?
Is there any situation where I would want to use Atom Pub over JSON?
There should be very little difference between ATOM and JSON on the semantic level with OData. Also most OData servers (WCF Data Services for sure) support both, so it's a choice of the client which one to use. As the blog post from Pablo mentions, to get the best payload size you should enable HTTP compression. It works great on both ATOM and JSON.
Reading JSON tends to be faster (XML parsing is kind of expensive), but that's if you're concerned with CPU consumption on the client. If I remember correctly, last time I saw the numbers, the compressed payload size for ATOM and JSON is not that different.
ATOM PUB is usually easier to consume in client which has available good XML or ATOM libraries and not JSON. And vice versa. But other than that, there should not be much of a difference.