Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I need to work with some systems that use JMESPath to search JSON. But I found that it is missing a lot of important features. Let's say, it is very hard to search string with pattern (like this). It does not support regular expression. It does not support case insensitive search. The proposal to add split function has been frozen since 2017 (like this and this). These features are all available to jq. So I want to know why systems like AWS S3 CLI, and Ansible use JMESPath instead of jq to query JSON?
It's not so much about the difference between JMESPath and jq as the different ways they are used.
Suppose you are querying a remote resource, the result is going to number in the millions of records, but you only care about a specific, much smaller subset of the records. You have two choices:
Have every record transmitted to you over the network, then pick out the ones you want locally
Send your filter to the remote resource, and have it do the filtering, only sending you the response.
jq is typically used for the former, JMESPath for the latter. There's no reason why the remote service couldn't accept a jq filter, or that you couldn't use a JMESPath-based executable.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
Why is JSON-formatted data stored in MongoDB? Is that the only format supported in MongoDB? What are the advantages of using JSON for storing records in MongoDB? What is the benefit of using JSON in Mong DB over other formats?
Actually Mongo uses BSON, that could represent the same things that JSON, but with less space. "JSON" (that is like the representation for human beings) have some properties useful in a NoSQL database:
No need for a fixed schema. You could just add whatever you want and it will be correct JSON.
There are parsers available for almost any programming language out there.
The format is programmer friendly, not like some alternatives... I'm looking at you, XML ¬¬.
Mongo needs to understand the data, without forcing a "collection schema". You don't need information about the object to reason about it, if it uses JSON. For example, you could get the "title" or "age" for any JSON document, just find that field. With other formats (eg. protocol buffers) thats not possible. At least without a lot of code...
(Added) Because Mongo is a database they want to do queries fast. BSON/JSON is a format that can meet that requirement AND the others at the same time (easily implementable, allow reflectioning about data, parsing speed, no fixed schema, etc).
(Added) Mongo reuses a Javascript engine for their queries, so it have all the sense in the world to reuse JSON for object representation. BSON is a more compact representation for that format.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I have CSV File's With Different Columns With Few Common Columns, We Are Currently Using Excel To Remove Unwanted Rows Clean The Data, and Generate Reports, I Am Thinking Of Using Elasticsearch As A Solution For Data Storage, Transformation, Load And Reporting.
Is Elasticsearch A Good Choice For This Use Case ?
Elastic Search is, as the name indicated, using to quick search. It is build upon Lucene and similar to another Apache project, Solr...
If you want to query the raw data or do some simple aggregation upon it. It is fine and you can also use Kibana to come up with some fancy GUI so your audience can interact with the data and you can even come up with some dashboard to demonstrate some basic staff. However, it is not a replacement of a data base.
If you want to update or join.. you had better use some data base ... sql + mongo or hive for big data.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I have a project which is using NodeJS and I have different entities for example, people and places.
I need the ability to find both types of entities by location together so what I was thinking of doing is having an index on a field called, type, for example, which would be either person or place and make use geospatial indexes, does this sound a good way to do this or is there a better way?
I will probably need a lot of joins too, so should I use MySQL alongside MongoDB and use MongoDB just for delivering the location based queries?
Thanks
This question is a poor fit for stackoverflow, but here's some radom bullet points:
PostgreSQL supports both joins and geospatial. I'd pick that first personally lacking other details warranting a different data store.
A totally valid option would be to keep people and places separate and query multiple collections as necessary. However, if you need to sort the results, then yes best to throw them in the same collection.
You could also keep people and places in separate mongodb collections but have a mapreduce job translate them into a locations collection for search purposes.
Generally, there are lots of ways to do this and the best one depends very much on the specific aspects of you application. Reads vs writes, data stability, data size, query load, etc, etc.
My broad word of advice is start with the most logical, easiest-to-follow, straightforward data organization (separate collections), and deviate from that when you understand the specific pain you have and how doing something more complicated or unusual will be an overall win.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I am using an email service API (can use either JSON or PHP) to send emails, and I want to store the replies into either MySQL or Mongo.
What do you think would be better for large amounts of email message storage, MySQL or Mongo?
It sort of depends on what you are doing, and what kind of metadata you want to store.
I have used both, but I have recently preferred to use MongoDB just because the way you store data (document-centric) is more conducive to the type of applications I work with than relational databases.
If what you want to store is a list of emails, and the replies to that email, then MongoDB might be a good way to go, since you could create a document structure like this:
{
'sender':'me#me.com,
'subject':'Hello world!',
'to':['tom#me.com','dick#me.com','harry#me.com'],
'body':'Hello, World, and stuff.',
'replies':[
{
'from':'tom#me.com',
'subject':'re: Hello World!',
'body':'Please stop sending me SPAM'
},
{ ... next reply...}
]
}
This makes it very nice to be able to query for specific messages and responses. Now, if you need to be able to query by individual users, the email addresses used, etc, and your primary use-case is going to be random queries from different directions, then MySQL might be better, since MongoDB doesn't really have support for joins.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I want to make a simple script that periodically performs SNMP GET and stores the output to MySQL database. It will be running on Windows.
What would be the least painful choice of programming language to write it in? Powershell, Java, …?
Python has SNMP (i.e. NET-SNMP) bindings. It will handle MySQL database as well. In that way you may use those APIs to create your own scanning application with SNMP communication channel and MySQL database storage of collected data. Of course there are tons of other solutions/combinations possible... however I recommend this one, because in such a simple case you don't have to engage heavy JVM.
Python has SNMP(Net-SNMP) binding and it's better that you can use RRD database to store output as it consists time interval itself.Python has RRD api also and you can handle easily