Does LogParser support JSON log files? I am working with an app that outputs simple JSON log files into a folder and I'm trying to run aggregate SQL style queries against the files in the folder.
The format of the files is simple:
{"f1":"value", "f2":NumericValue, "f3":"DateValue", etc...}
You can try out Musoq which already has a plugin that lets you treat json as queryable source.
Unfortunately not. LogParser was written when XML was the cool thing :-)
But you can always write your COM extension and LogParser will be able to query JSON files as well.
Related
I have created one file .testcaferc.json that contains all configuration information like browser name, specs, timeouts etc. I want to fetch the configuration data from file so that I have to change the information at only one place
I want to store all these information in single file, I tried, js, json and array. But I can not import all above format files in my .testcaferc.json, when I press Alt+F8 I see the error "Expected a JSON object, array or literal"
Is there any way I can import json, array or js data in .testcaferc.json?
Thanks in advance!!
The JSON format doesn't support any import directives. The TestCafe configuration file (.testcaferc.json) is a simple JSON file. So, the TestCafe configuration file doesn't support such functionality.
To achieve your goal, you can transform the existing .testcaferc.json file before test running: load data from various sources and add/replace values for the appropriate data fields.
Also, there is a suggestion in the TestCafe GitHub repository, which will make your scenario easier to implement. Track it to be notified about its progress.
I have noticed there is a feature in web interface of ArangoDB which allows users to Download or Upload data as JSON file. However, I find nothing similar for CSV exporting. How can an existing Arango DB collection be exported to a .csv file?
If you want to export data from ArangoDB to CSV, then you should use Arangoexport. It is included in the full packages as well as the client-only packages. You find it next to the arangod server executable.
Basic usage:
https://docs.arangodb.com/3.4/Manual/Programs/Arangoexport/Examples.html#export-csv
Also see the CSV example with AQL query:
https://docs.arangodb.com/3.4/Manual/Programs/Arangoexport/Examples.html#export-via-aql-query
Using an AQL query for a CSV export allows you to transform the data if desired, e.g. to concatenate an array to a string or unpack nested objects. If you don't do that, then the JSON serialization of arrays/objects will be exported (which may or may not be what you want).
The default Arango install includes the following file:
/usr/share/arangodb3/js/contrib/CSV_export/CSVexport.js
It includes this comment:
// This is a generic CSV exporter for collections.
//
// Usage: Run with arangosh like this:
// arangosh --javascript.execute <CollName> [ <Field1> <Field2> ... ]
Unfortunately, at least in my experience, that usage tip is incorrect. Arango team, if you are reading this, please correct the file or correct my understanding.
Here's how I got it to work:
arangosh --javascript.execute "/usr/share/arangodb3/js/contrib/CSV_export/CSVexport.js" "<CollectionName>"
Please specify a password:
Then it sends the CSV data to stdout. (If you with to send it to a file, you have to deal with the password prompt in some way.)
Im trying out the MarkLogic Java API and would want to bulk upload some files with the extension .csv
I'm not sure what to use, since the Java API only supports JSON, XML, and TXT files.
How do I batch upload files using the MarkLogic Java api? Do i convert everything to JSON?
Do i convert everything to JSON?
Yes, that is a common way to do it.
If you would like additional examples of how you can wrangle CSV with the Java Client API, check out OpenCSVBatcherExample and JacksonDatabindTest.testDatabindingThirdPartyPojoWithMixinAnnotations. The first demonstrates converting the csv to XML and using a custom REST extension. The second example (well, unit test...) demonstrates converting the csv to JSON and using the batch upload (Bulk Writes) capabilities Justin linked to.
If you have CSV files on your filesystem, I’d start with mlcp, as suggested above. It will handle all of the parsing and splitting into multiple transactions/batches for you. Take a look at the mlcp documentation for more details and some example configurations.
If you’d like more control over the parsing and splitting logic than mlcp gives you out-of-the-box or you’re getting CSV from some other source (i.e. not files on the filesystem), you can use the Java Client API. The Java Client API allows you to efficiently write batches using a WriteSet. Take a look at the “Bulk Writes” example.
According to your reply to Justin, you cannot use MLCP because it is command line and you need to integrate it into a web portal.
Well, MLCP is released as open cource software under the Apache2 licence. So if you are happy with this licence, then you have the source to integrate.
But what I see as your main problem statement is more specific:
How can I create miltiple XML OR JSON documents from a CSV file [allowing the use of the java API to then upload them as documents in MarkLogic]
With that specific problem statement:
1) have a look at SplitDelimitedTextReader.java from the mlcp source
2) try some java libraries for this purpose such as http://jsefa.sourceforge.net/quick-tutorial.html
I'm working on a specific project where external data provided by external providers is to be indexed on our ElasticSearch Engine.
The data is provided as XML flat files.
The idea here is to script something out that reads each file, parse it and launch as many HTTP POST as needed for each one of them.
Is there a simpler way to do this? something like uploading the XML file that gets indexed automatically without any script?
You can use logstash with an xml filter to do this. Takes a bit of work to get setup the first time, but it's the most straightforward way to do it.
How to convert the BSON into JSON from MongoDB.
I inserted documents into MongoDB then I get the following error.
This what I get
C:\mongodb\bin>bsondump collection.bson> collection.json error: boost::filesystem::file_size:
The system cannot find the filespecified:"collection.bson"
please use the absolute directory of collection.bson file or see whether the file is in C:\mongodb\bin or not
Firstly, you have to run the mongodump, which makes the bson file. It creates a dump folder, and it makes for every database an other folder. After that you have to run bsondump coll.bson > coll.json.