I am using HDF5 data layer. The layer will read the image and its label (from the list.txt) and feed them to the network.
layer {
name: "data"
type: "HDF5Data"
top: "data"
top: "label"
include {
phase: TRAIN
}
hdf5_data_param {
source: "./list.txt"
batch_size: 10
shuffle: true
}
}
Because my list.txt contain some image with the empty label (means black image). I want to check if the empty label is selected during the training. In CAFFE, can we do it?
Related
We have a couple of legacy applications we're migrating to ec2 and these use a bunch of application configuration parameters. I need to be able to store each config as an individual parameter per application.
I'm trying the following but clearly not doing it right as it appends all values to a single parameter per application:
locals {
application = {
"application1" = { app_shortcode = "app1"},
"application2" = { app_shortcode = "app2"}
}
resource "aws_ssm_parameter" "application_parameters" {
for_each = local.application
name = each.key
value = jsonencode(file("${path.module}/${each.key}/ssm_param.json"))
}
my app1's ssm_param.json is something like
{
"app1_config1": "config_value_1",
"app1_config2": "config_value_2",
"app1_config3": "config_value_3"
}
and app2's ssm_param.jsonis
{
"app2_config_a": "config_value_a",
"app2_config_b": "config_value_b",
"app2_config_c": "config_value_c"
}
The current output is a single parameter like this for each application:
"{\r\n \"app2_config_a\": \"config_value_a\",\r\n \"app2_config_b\": \"config_value_b\"\r\n, \r\n \"app2_config_c\": \"config_value_c\"\r\n}"
Looking for suggestions please.
I solved this by using a slightly different approach (not quite the same as my initial one but this works for me for now):
used a ssm_params.yaml as below (the project team was kind enough to give me the config settings as yaml output)
parameter:
app1:
name: app1_config1
description: "application config test"
type: "String"
value: "some_randoM_value"
app1:
name: app_config2
description: "another test"
type: "SecureString"
value: "some_random123_value###"
app2:
name: app_config_2
description: "config test"
type: "String"
value: "some_randoM_value_2"
locals {
params = yamldecode(file("${path.module}/ssm_params.yaml"))
}
resource "aws_ssm_parameter" "app_params" {
for_each = local.params.parameter
name = each.value.name
type = each.value.type
value = each.value.value
}
I would like to extract the scores from the GoogleNet network from caffe models but I don't quite understand which layers keep the scores.
I get:
Check failed: feature_extraction_net->has_blob(blob_names[i]) Unknown feature blob name loss3/top-5 in the network ./train_val.prototxt
Any suggestion?
The layer loss3/top5 is an Accuracy layer. I will assume that this is want you meant by scores.
By default, you will have something like this
layer {
name: "loss3/top-5"
type: "Accuracy"
bottom: "loss3/classifier"
bottom: "label"
top: "loss3/top-5"
include {
phase: TEST
}
accuracy_param {
top_k: 5
}
}
If you want to have this layer in training, you can remove or comment out (#) the include section.
I am using caffe and the lmdb database to feed my data to the network. However, I have two different lmdbs for input and ground_truth since my ground_truth are images as well. Is it possible to use shuffle anyways? If yes, can I just set shuffle: true for both lmdbs as param?
layer {
name: "data"
type: "Data"
top: "data"
include {
phase: TRAIN
}
transform_param {
mean_value: X
}
data_param {
source: "..."
batch_size: X
backend: LMDB
}
If you use layer type "Data" you can't use shuffle as there is no shuffle parameter in data_param.
As for layer type "ImageData" you can't use lmdb as data source as source file should be a text file with image address and label. But it has shuffle parameter. If you look inside image_data_layer.cpp you'll find if shuffle is true then image sources are shuffled in each epoch using Fisher–Yates algorithm. If you use two different ImageData layer then ShuffleImages() will be called for each of them and it is unlikely that two shuffle will generate the same sequence. So you can't use shuffle in any of these two ImageData layer.
I tried to make use of the test_state functionality in caffe solver while training. To implement this I added the following code to solver.prototxt
test_state: { stage: 'test-on-testSet0' }
test_iter: 726
test_state: { stage: 'test-on-testSet1' }
test_iter: 363
Then I modified the train_val.prototxt like this:
layer {
name: "data"
type: "ImageData"
top: "data"
top: "label"
include {
phase: TEST
stage: "test-on-testSet0"
}
transform_param {
mirror: false
scale: 0.0039215684
}
image_data_param {
source: "./set0.lst"
batch_size: 1
}
}
layer {
name: "data"
type: "ImageData"
top: "data"
top: "label"
include {
phase: TEST
stage: "test-on-testSet1"
}
transform_param {
mirror: false
scale: 0.0039215684
}
image_data_param {
source: "./set0.lst"
batch_size: 2
}
}
It has to be noted that both the test cases are ideally the same and the test runs on the complete set of images present in the ./set0.lst file.
Still while training using build/tools/caffe, the results of the accuracy printed for both the test states are not identical.
The accuracy layers are connected correctly too.
What could be the reason for this mismatch?
I was able to fix the issue by using the same batch_size for all the test_states. Looks like caffe expects that all the test cases have the same batch_size.
Hope this answer might help someone in the future.
Btw, I guess, this could be issued as a bug to the caffe community. I was facing this issue with the latest commit of caffe (df412ac).
I would like to have a ListModel-like structure to display inputs of a simple state machine. Each input might consist of several strings/ints. So I need each item of the ListModel to be able to store a list of data (strings with names of the input's parameters, or dictionaries with strings etc). At the moment I cannot append an item with a list property to a ListModel.
So the ListModel looks like this:
ListView {
anchors.fill: parent
model: ListModel {
id: listModel
}
delegate: Text {
text: inputs[0]['name']
}
}
And when the state changes I want to update the model and append elements like this:
var state = {
name: "abcd",
inputs: [{name: 'a'}, {name: 'b'}, {name: 'c'}]
}
listModel.append(state);
Current version of the code returns error TypeError: Cannot read property 'name' of undefined. It does not see the list.
According to this question there might be issues with using lists in the items of ListModel. But it seems irrelevant to my case. Maybe I need to use lists and dicts differently in QML, maybe I had to write text: inputs[0].name in delegate (which I tried) or something else (suggestions?).
Could someone suggest how to make a more or less complex item (basically, it is standard JSON) in a ListModel? It is not clear, since documentation and blogs/questions deal with strings all the time. Is there some helpful documentation which I missed? What are good practices to do it in QML? Should I use some custom objects?
List data can be added to a ListElement, according to the documentation and as you correctly did in your imperative code. However, nested roles are not really arrays. They are ListModels themselves. That's because, by design, QML does not produce a notification if an element of an array changes, which would be a show-stopper in a model-view-delegate setting.
Since the nested role is a model, you can use model's functions. For instance, this example works fine:
import QtQuick 2.5
import QtQuick.Window 2.2
Window {
id: window
width: 600
height: 400
visible: true
ListView {
anchors.fill: parent
model: ListModel {
id: listModel
}
delegate: Text {
text: name + inputs.get(index % inputs.count).name // accessing the inner model
}
}
MouseArea {
anchors.fill: parent
onClicked: {
var state = {
name: "abcd",
inputs: [{name: 'a'}, {name: 'b'}, {name: 'c'}]
}
listModel.append(state);
}
}
}
According to your question, the input is plain JSON. In that case, consider the usage of JSONListModel in place of ListModel. It exposes a set of API which matches XMLListModel, via JSONPath, and could possibly represent the perfect solution for your scenario.