In the Mechanical Turk Command Line Tools external_hit example, a CSV titled external_hit.input is uploaded along with external_hit.question and external_hit.properties. This CSV contains values that will be injected into the ExternalURL element in external_hit.questions.
I would like to use boto.mturk to construct the same example. How do I upload external_hit.input when creating a new HIT with boto?
Related
Context:
For a data pipeline we need to ingest excel spreadsheets directly into foundry (arriving via email). In order to avoid any manual handling error, we'd like to build a small slate app that basically just uploads an excel sheet and automatically appends it to an existing dataset (given schema, headers, etc.).
Unfortunately, there is very little documentation on the "File Import" widget or the API that gets called when drag and dropping a file into a folder.
Idea: Is there a way of uploading a file with slate? Could this file then be added to a dataset, similarly as with the prompt that opens when dropping it into a folder?
You actually don't have to build a Slate app to do this! Datasets that are made up of underlying .csv files support new additions of files directly.
Note: All of the following screenshots are from the dataset preview page.
For example, the following dataset I created from 4 .csv files:
And I can click on the Import button in the top right to add in more files (with the same schema, or not. Depends on if you want to strictly adhere to your applied schema.
If you have already applied a schema, you can also simply Import new files on top of the dataset, but the schemas of the files must exactly match those already present, otherwise your dataset will fail when attempted to be read.
How can I make JMeter read the second sheet of my CSV?
I want to use CSV Data Set Config.
Normally, it reads the first line of the first sheet but is there any way to be a bit more flexible?
CSV file format doesn't have "sheets", it is a normal plain text file using delimiters in order to represent structured data.
If you are trying to get data from i.e. Microsoft Excel file type - unfortunately you won't be able to do it using CSV Data Set Config. The easiest would be exporting data as separate plain-text CSV files.
If you don't have the possibility to do the export you still can access the data from Excel files but it will be a little bit more tricky as you will have to use JSR223 Test Elements, Groovy language and Apache POI libraries
More information:
Busy Developers' Guide to HSSF and XSSF Features
How to Extract Data From Files With JMeter
Currently you can use CSV Data Set Config for that, you should add external code for example using Apache Commons CSV,
Download the jar file and place it in JMETER_HOME lib folder, and then write the code in JSR223 Element.
Examples exists, code for get second record:
Reader in = new FileReader("path/to/file.csv");
Iterable<CSVRecord> records = CSVFormat.RFC4180.parse(in);
// go to next record
records.next();
CSVRecord secondRecord = records.next();
//columnOne = secondRecord.get(0);
I am trying to export all the plots of my NetLogo model after simulation runs in a csv format with the primitive export-all-plots.
I haven't found yet the way to open this csv file with an external reader in order to get more clear plots. I tried with gnuplot but it looks like it's not able to open the csv format created with NetLogo:
"export-plots data (NetLogo 5.0.5)"
^
"C:\results\interface.csv", line 1: invalid command
How can I open csv plots with an external reader?
There are two complicating factors about NetLogo's plot export format. First, there's a three line header at the beginning (plus an empty line after) that just gives information about the model and when the data was generated. Next, there's data about the model settings, the plot state (pen colors and such). Finally, there's the data itself, which itself is somewhat complicated by the fact that you can have multiple pens per plot. So I'm not surprised gnuplot couldn't read it as is.
The table's are quite easy to use in GUI spreadsheet application, like Excel, LibreOffice's Calc, or Gnumeric. You can just select the data you want and generate the plots.
To do this at the command line, I'm afraid you might have to write a script to read it in. This should be pretty easy in something like Python or R. Just skip the metadata lines, and use a CSV parser to read in the rest.
You might also try using BehaviorSpace to generate the data, but make sure to use the table output. It let's you generate the data from many runs at once, and the format is a little more consistent. There are still 6 lines of metadata at the top, but you can just delete that. I believe this is more the standard practice in NetLogo.
I have a custom content type (product) containing a title, some text and a file (a pdf file).
A lot of products have to be imported to the Drupal CMS. I discovered the plugin "Feeds" which seems to fulfill my requirements.
I managed to import a csv files containing a title and some text.
Now in my csv file I have a column containing a path to all the pdf files (like C:\mypdfs\product1.pdf). How can I manage that those pdf files are imported by Feeds? In "Mapping" configuration I'm not sure what target I have to select for those column which contains the path to a local file.
Using Feeds Tamper module you can manipulate the value of imported data for one target. Here you can build a custom tamper (See How) and use it to process the retrieved value (file path), using file_get_contents to get the file from imported path and file_save_data to save a file in Drupal, getting a field object that you can attach to an entity (this could help).
I'm using JFreeChart to generate line graphs from a simple array of integers.
However, I'd like to use a csv file for the input of the graph. Are there any applications which do it automatically? Also, my csv file will new entries appended every 3 seconds. How can I generate dynamic graphs? I will have to use these graphs in a swing application. Thanks!
I've had good luck with org.h2.tools.Csv, part of H2 Database. You might also look at org.jfree.data.io.CSV, "A utility class for reading CategoryDataset data from a CSV file."