I have a post endpoint that will accept some parameters like file and name in order for me to upload a file. I want to achieve this using Knime but I'm not sure how I can pass the file data to the post node.
What would be a minimal Knime flow that can read the data from a local file and
prepare it for the post node?
File uploads are typically done with so-called “multipart encoding”.
You can use the HTTP Retriever and its companion node Multipart Encoded HTTP Entity Creator from the Palladian package for that. Palladian is a versatile extension which provides nodes for text classification, HTML parsing, HTTP requests, and geo data extraction which is freely available for the free KNIME versions.
An example workflow how to do a file upload using the mentioned nodes can be found at the following URL on my NodePit Space:
https://nodepit.com/workflow/com.nodepit.space/qqilihq/public/Palladian/HttpRetriever_Multipart_Example.knwf
Related
I built an api with next.js.
I use a JSON file as a data source. I import it as a module. But if the content of the JSON changes, it still shows the first content, the same, when i started the server.
Is there a way to avoid caching JSON with import?
I need to get the JSON content, but also the updates in the JSON file, without restarting my api.
If your Server returns the JSON files with a specific File-Extension like .json you could try to turn off the caching for those file-types:
Here is an example for ngnix-servers
Here is an example for apache-servers
Another possibility is to load the JSON via Javascript where you add some random parameter to the Query-Params of the URL
Here is the Example
I have created one file .testcaferc.json that contains all configuration information like browser name, specs, timeouts etc. I want to fetch the configuration data from file so that I have to change the information at only one place
I want to store all these information in single file, I tried, js, json and array. But I can not import all above format files in my .testcaferc.json, when I press Alt+F8 I see the error "Expected a JSON object, array or literal"
Is there any way I can import json, array or js data in .testcaferc.json?
Thanks in advance!!
The JSON format doesn't support any import directives. The TestCafe configuration file (.testcaferc.json) is a simple JSON file. So, the TestCafe configuration file doesn't support such functionality.
To achieve your goal, you can transform the existing .testcaferc.json file before test running: load data from various sources and add/replace values for the appropriate data fields.
Also, there is a suggestion in the TestCafe GitHub repository, which will make your scenario easier to implement. Track it to be notified about its progress.
With the closure of MyJson.com, where can a developer upload a json file to be parsed? I have completed my file with Json Feed Manager and uploaded the file to GitHub, but the ability to read to use the URL as a raw does not work any longer. I'm thinking this was turned into a paid feature vs. free. Some of the other Roku developers have hinted the best suggestion is to acquire a paid webhost and make a directory for json to be uploaded into it and point the Direct Publisher to the source.
https://community.roku.com/t5/Roku-Direct-Publisher/bd-p/roku-direct-publisher
I have completed my file with Json Feed Manager and uploaded the file to GitHub, but the ability to read to use the URL as a raw does not work any longer.
I don't believe this is the case. You can still access raw content from GitHub through the domain https://raw.githubusercontent.com
Template:
https://raw.githubusercontent.com/<username>/<repository>/<branch>/<path-to-file>
Example:
https://raw.githubusercontent.com/pomber/covid19/master/package.json
I have a directory with CSV files. each file contains a list of GET requests I'd like to make with JMeter. What I'd like to do is read all the files in a directory, and then loop through each CSV to send the requests in JMeter. The number of files isn't consistent so I don't want to hard code the file names into CSV samplers.
So in effect I'd like to read all the files in the directory and store the files in an array variable. The loop through the array and send the CSV file to the CSV sampler which will in turn read the CSV file and pass the content to an HTTP Request sampler to send the GET requests.
I created a beanshell script to read the files in the directory and store them in an array, but when I try to pass this to the CSV config element, I get errors stating the variable doesn't exist.
I've tried another beanshell script to read the file and pass the lines to an HTTP request Sampler as a variable, but the issue was, it would store all the file contents in memory per thread.
I'd like to know the best approach to read the files, send the requests and use the response data to generate reports
You will not be able to populate CSV Data Set config using Beanshell as CSV Data Set Config is a Configuration Element and according to Execution Order user manual chapter Configuration Elements are executed before anything else.
Since JMeter 3.1 you should not be using Beanshell, it is recommended to switch to JSR223 Elements and Groovy language
I would recommend going for Directory Listing Config plugin, it scans the provided folder (in your case with CSV files) and stores the found paths to files into a JMeter variable
So you can use the Directory Listing Config in combination with __StringFromFile() or __CSVRead() functions and that should be more or less good way of implementing your requirements.
Im trying out the MarkLogic Java API and would want to bulk upload some files with the extension .csv
I'm not sure what to use, since the Java API only supports JSON, XML, and TXT files.
How do I batch upload files using the MarkLogic Java api? Do i convert everything to JSON?
Do i convert everything to JSON?
Yes, that is a common way to do it.
If you would like additional examples of how you can wrangle CSV with the Java Client API, check out OpenCSVBatcherExample and JacksonDatabindTest.testDatabindingThirdPartyPojoWithMixinAnnotations. The first demonstrates converting the csv to XML and using a custom REST extension. The second example (well, unit test...) demonstrates converting the csv to JSON and using the batch upload (Bulk Writes) capabilities Justin linked to.
If you have CSV files on your filesystem, I’d start with mlcp, as suggested above. It will handle all of the parsing and splitting into multiple transactions/batches for you. Take a look at the mlcp documentation for more details and some example configurations.
If you’d like more control over the parsing and splitting logic than mlcp gives you out-of-the-box or you’re getting CSV from some other source (i.e. not files on the filesystem), you can use the Java Client API. The Java Client API allows you to efficiently write batches using a WriteSet. Take a look at the “Bulk Writes” example.
According to your reply to Justin, you cannot use MLCP because it is command line and you need to integrate it into a web portal.
Well, MLCP is released as open cource software under the Apache2 licence. So if you are happy with this licence, then you have the source to integrate.
But what I see as your main problem statement is more specific:
How can I create miltiple XML OR JSON documents from a CSV file [allowing the use of the java API to then upload them as documents in MarkLogic]
With that specific problem statement:
1) have a look at SplitDelimitedTextReader.java from the mlcp source
2) try some java libraries for this purpose such as http://jsefa.sourceforge.net/quick-tutorial.html