I have a file which contains a header(specific to a file used by octave) and I have to read the matrix contained in the file. The idea is that I have a function and the file location is stored in a string let's say location. If I try to do load(location) I get some parse error and that matrix will not be load.
function [extract] = read(location)
matrix = ?;
end
Related
I have a view that should generate a temporary JSON file and save this TempFile to the database. The content to this file, a dictionary named assets, is created using DRF using serializers. This file should be written to the database in a model called CollectionSnapshot.
class CollectionSnapshotCreate(generics.CreateAPIView):
permission_classes = [MemberPermission, ]
def create(self, request, *args, **kwargs):
collection = get_collection(request.data['collection_id'])
items = Item.objects.filter(collection=collection)
assets = {
"collection": CollectionSerializer(collection, many=False).data,
"items": ItemSerializer(items, many=True).data,
}
fp = tempfile.TemporaryFile(mode="w+")
json.dump(assets, fp)
fp.flush()
CollectionSnapshot.objects.create(
final=False,
created_by=request.user,
collection_id=collection.id,
file=ContentFile(fp.read(), name="assets.json")
)
fp.close()
return JsonResponse({}, status=200)
Printing assets returns the dictionary correctly. So I am getting the dictionary normally.
Following the solution below I do get the file saved to the db, but without any content:
copy file from one model to another
Seems that json.dump(assets, fp) is failing silently, or I am missing something to actually save the content to the temp file prior to sending it to the database.
The question is: why is the files in the db empty?
I found out that fp.read() throws content based on the current pointer in the file. At least, that is my understanding. So after I dump assets dict as json the to temp file, I have to bring back the cursor to the beggining using fp.seek(0). This way, when I call fp.read() inside file=ContentFile(fp.read(), ...) it actually reads all the content. It was giving me empty because there was nothing to read since the cursor was at the end of the file.
fp = tempfile.TemporaryFile(mode="w+")
json.dump(assets, fp)
fp.flush()
fp.seek(0) // important
CollectionSnapshot.objects.create // stays the same
I was trying to use the method found here (see most up-voted answer):
Google Apps Script Fastest way to find a row?
I currently use this while it does work I wanted to try the above linked method yet when I replace the below code
function AutoPopulate (evalue)
{
//uses google drive file irretator reads in JSON file and parses it to a Javascript object that we can work with
var iter = DriveApp.getFilesByName("units.json");
// iterate through all the files named units.json
while (iter.hasNext()) {
// define a File object variable and set the Media Tyep
var file = iter.next();
var jsonFile = file.getBlob().getDataAsString();
// log the contents of the file
//Logger.log(jsonFile);
}
var UnitDatabase = JSON.parse(jsonFile);
//Logger.log(UnitDatabase);
//Logger.log(UnitDatabase[1027]);
return UnitDatabase[evalue];
}
WITH THIS CODE:
function AutoPopulate (evalue)
{
//this method did not work for me but should have according to stackflow answer linked above I am trying to understand why or how I can find out why it may have thrown an error
var jsonFile = DriveApp.getFilesByName("units.json").next(),
UnitDatabase = UnitDatabase.getBlob().getDataAsString();
return UnitDatabase[evalue];
}
I get an error in the excecution indicating that there is a % at postion 0 in the JSON, between the methods I dont alter the JSON file in anyway so I dont understand why does the top method work but the bottom one does not?
For further information the idea behind the code is that I have a list of Unit numbers and model numbers that are in a spreadsheet. I then convert this to a JSON file, this however is only done when a new unit is added to the fleet. As I learned one can parse a whole JSON file into a javascript object which makes working with the data set much faster. This javascript object is used so that when a user enters a UNIT# the MODEL# is auto populated based on the JSON file.
I cannot share the JSON file as it contains client information.
Your code does not work for two reasons:
You have a typo in the line UnitDatabase = UnitDatabase.getBlob()... - it should be UnitDatabase = jsonFile.getBlob()...
If you want to retrieve a nested object from a json file - you need to parse the JSOn - otherwise it is considered a string and you can not access the nested structure
Modified working code:
function AutoPopulate2 (evalue)
{
var jsonFile = DriveApp.getFilesByName("units.json").next();
var UnitDatabase = JSON.parse(jsonFile.getBlob().getDataAsString());
return UnitDatabase[evalue];
}
Mind that this code will only work if you have a "units.json" file on your drive and if evalue is a valid 1st-level nested object of this json.
I have a customized layer with certain parameters like kernel size,pad info etc.. among this I have array parameter which is of type bool/int. I need to add this array parameter in .caffemodel file after processing. How to link/Dump an array into caffemodel file?
I have included array parameter in .proto file like below.
{
... //other parameters
repeated bool/int <varibale_name> [packed = true]
}
Is it possible to create blob of type bool?
You can do it through py-caffe net surgery -
net.params[learnable_layer_name].add_blob()
new_blob_index = len(net.params[learnable_layer_name]) - 1
net.params[learnable_layer_name][new_blob_index].reshape(desired_shape)
net.params[learnable_layer_name][new_blob_index].data[:] = new_data_to_insert
HTH.
Say I have a function that reads a .txt file and creates arrays based on the columns of the data within that file. What I have right now inside the function looks like:
data = open("some_file_name.txt","r")
But if I want to change the .txt file that the function reads I have to manually go into the code and type in the new file name before running it again. Instead, how can I pass any file name to the function so it looks like:
my_function(/filepath/some_file_name.txt):
data = open("specified_file_name.txt","r")
I think you want
def my_function(filepath):
data = open(filepath, "r")
...
and then
my_function("/filepath/some_file_name.txt")
or better:
def my_function(data):
...
and then
with open("/filepath/some_file_name.txt", "rb") as data:
my_function(data)
The latter version lets you pass in any file-like object to my_function().
Update: if you want to get fancy and allow file names or file handles:
def my_func(data):
if isinstance(data, basestring):
with open(data, 'rb') as f:
return my_func(f)
...
I need to test a various links of a site (no need to login) with 100's of users and loop it for some number of times using JMeter. I want to put those links in a "CSV file", so that all the links to be tested are read from file.
How do I accomplish this task?
Prepare kind of csv-file with list of your test-params and use it to parametrize your test-samplers, using at least the following:
CSV Data Set Config
Look into the following links for details:
How to get Jmeter to use CSV data for GET parameters?
Use jmeter to test multiple Websites
use csv parameters in jmeter httprequest path
Force a thread to use same input line when using CSV Data Set Config
Jmeter functions:
__CSVRead,
__StringFromFile.
Variables From CSV sampler from jmeter-plugins.
1. Prepare your test-urls in csv-file, e.g. in the following format:
url1
url2
...
urlN
Ensure that test-URLs don't contain http:// prefix (as per HTTP Request params -> Server).
2. Use schema for your script as below:
CSV Data Set Config:
Filename: [path to your csv-file with test-urls]
Variable Names: testURL
Recycle on EOF?: True
Stop thread on EOF?: False
Sharing mode: Current thread
Thread Group:
Number of Threads: N
Loop Count: M
HTTP Request // your http call
Server Name or IP: ${testURL} // use variable with extracted URL
This will start N users, each users will read M entries from list of test-urls. If M > number of entries in list of test-urls then user will recycle the list on EOF.
In one of the comments, it's mentioned that you can't read the CSV more than once per loop. You can go and have multiple threads, each reading the CSV file once, but then the file is close and won't be read on the next loop. Also, if you set the CSV to recycle, then CSV file is read over and over again indefinitely. So the question becomes how do you loop a CSV file a certain number of times as opposed to indefinitely?
I posted my answer to that in another post (https://stackoverflow.com/a/64086009/4832515), but I'll copy & paste it incase that link doesn't work in the future.
I couldn't find a simple solution to this. I ended up using beanshell scripts, which let you use code very similar to java to do some custom stuff. I made an example JMeter project to demonstrate how to do this (yes it's ridiculously complicated, considering all I want to do is repeat the CSV read):
Files:
my file structure:
JMeterExample
|
⊢--JMeterTests.jmx // the JMeter file
⊢--example.csv // the CSV file
contents of my CSV:
guest-id-1,"123 fake street",
guest-id-2,"456 fake street",
guest-id-3,"789 fake street",
so in this thread group, I'm going to just have 1 user, and I'll loop 2 times. I intend to send 1 request per CSV line. So there should be 6 requests sent total.
Thread Group
User Defined Variables
This is kind of optional, but the filepath is subject to change, and I don't like changing my scripts just for a change in configuration. So I store the CSV filename in a "User Defined Variables" node.
If you are storing the CSV file in the same directory as your JMeter test, you can just specify the filename only.
If you are saving the CSV in a folder other than the directory containing your JMeter file, you will need to supply an absolute path, and then slightly modify the beanshell script below: you'll need to comment out the line that loads the file relatively, and comment in the line that loads from an absolute path.
BeanShell Sampler to parse and store CSV lines
Add a Beanshell Sampler which will basically take in a path, and parse & store each line as a variable. The first line will be stored as a variable called csv_line_0, the 2nd line will be csv_line_1 and so on. I know it's not a clean solution but... I can't find any clean simple way of doing this clean simple task. I copied and pasted my code below.
import org.apache.jmeter.services.FileServer;
import java.text.*;
import java.io.*;
import java.util.*;
String temp = null;
ArrayList lines = new ArrayList();
BufferedReader bufRdr;
ArrayList strList = new ArrayList();
// get the file
try {
// you can use this line below if your csvFilePath is an absolute path
// File file = new File(${csvFilePath});
// you can use this line below if your csvFilepath is a relative path, relative to where you saved this JMeter file
File file = new File(org.apache.jmeter.services.FileServer.getFileServer().getBaseDir() + "/" + ${csvFilePath});
if (!file.exists()) {
throw new Exception ("ERROR: file " + filename + " not found");
}
bufRdr = new BufferedReader(new InputStreamReader(new FileInputStream(file), "UTF8"));
} catch(Exception e){
log.error("failed to load file");
log.error(e.getMessage());
return;
}
// For each CSV line, save it to a variable
int counter = 0;
while(true){
try{
temp = bufRdr.readLine();
if(temp == null || temp.equals("<EOF>")){
break;
}
lines.add(temp);
vars.put("csv_line_" + String.valueOf(counter), temp);
counter++;
} catch(Exception e){
log.error("failed to get next line");
log.error(e.getMessage());
break;
}
}
// store the number of CSV lines there are for the loop counter
vars.put("linesCount", String.valueOf(lines.size()));
Loop Controller
Add a Loop Controller that loops once for each CSV line. ${linesCount} is a count of the number of CSV lines and is calculated from the above beanShell script.
Beanshell script to extract data from current CSV Line
This script will run once per CSV line. It will go and grab the current line, and parse out whatever data is on it. You'll have to modify this script to get the data you want. In my example, I only had 2 columns, where column 1 is a "guestId", and column 2 is an "address".
__jm__loopController__idx is a variable JMeter defines for you, and is the index of the loop controller. The variable name is __jm__{loop controller name}__idx.
String index = vars.get("__jm__loopController__idx");
String line = vars.get("csv_line_" + index);
String [] tokens = line.split(",");
vars.put("guestId", tokens[0]);
vars.put("address", tokens[1]);
Http request sampler
Here's the HTTP request that's using the data extracted.
result
When running this, as desired, I end up sending 6 http requests over to the endpoint I defined.