When I try to convert a model from Caffe to Core ML model with coremltools, I get the following:
================= Starting Conversion from Caffe to CoreML ======================
Layer 0: Type: 'Data', Name: 'data'. Output(s): 'data', 'label'.
WARNING: Skipping Data Layer 'data' of type 'Data'. It is recommended to use Input layer for deployment.
Layer 1: Type: 'Split', Name: 'label_data_1_split'. Input(s): 'label'. Output(s): 'label_data_1_split_0', 'label_data_1_split_1'.
Layer 2: Type: 'Convolution', Name: 'conv1'. Input(s): 'data'. Output(s): 'conv1'.
Layer 3: Type: 'Slice', Name: 'slice1'. Input(s): 'conv1'. Output(s): 'slice1_1', 'slice1_2'.
Layer 4: Type: 'Eltwise', Name: 'etlwise1'. Input(s): 'slice1_1', 'slice1_2'. Output(s): 'eltwise1'.
Traceback (most recent call last):
File "test.py", line 2, in <module>
coreml_model = coremltools.converters.caffe.convert('_iter_3560000.caffemodel')
File "/Users/zfh/Desktop/face_verification_experiment/model/python27/lib/python2.7/site-packages/coremltools/converters/caffe/_caffe_converter.py", line 142, in convert
predicted_feature_name)
File "/Users/zfh/Desktop/face_verification_experiment/model/python27/lib/python2.7/site-packages/coremltools/converters/caffe/_caffe_converter.py", line 187, in _export
predicted_feature_name
RuntimeError: Unsupported option 'Max' for the parameter 'operation' in layer 'etlwise1' of type 'Elementwise' during caffe conversion.
This is the code I am using:
import coremltools
coreml_model = coremltools.converters.caffe.convert(('_iter_3560000.caffemodel', 'LCNN_deploy.prototxt'))
coreml_model.save('_iter_3560000.mlmodel')
Any ideas what the problem is? Thank you very much!
As the error message says, the problem is that the Max operation in an Eltwise layer is not supported by the coremltools. Core ML only supports a limited number of layers.
However... it seems like maybe you're trying to convert the .prototxt that was used for training (even though the filename is LCNN_deploy.prototxt). Are you sure this is the correct deploy.prototxt?
Recently, I extract the convert tools of caffe2mlmodel from the coremltools, its the c++ implemention.
First of all,you need to know this tool supported caffe layer, defined in caffe.proto(include in caffeconverter directory)
And, then, open the caffe.proto, you can locate at the message LayerParameter, like below:,. You can find the supported caffe layer.
caffe.proto in caffeconverter of coremltools
Finally, if you want the custom caffe layer, just add adapt the caffe.proto , and learn the Core ML mode protobuf specification(https://apple.github.io/coremltools/coremlspecification/#)
Related
Starting in macOS 12 (Monterey), the system apparently writes crash file as .ips files, instead of the traditional .crash file format.
The file appears to contain JSON data:
{"app_name":"Helper","timestamp":"2021-10-30 18:49:32.00 +0100","app_version":"3.0.0(66) beta","slice_uuid":"673198dd-94ac-31a7-9e81-09fe6c781255","build_version":"3.0.0.66","platform":1,"bundleID":"com.dislt.helper","share_with_app_devs":0,"is_first_party":0,"bug_type":"309","os_version":"macOS 12.0.1 (21A559)","incident_id":"CC03C2EC-C1D4-4F6E-AA1F-6C4EC555D6B8","name":"Helper"}
{
"uptime" : 91000,
"procLaunch" : "2021-10-30 18:49:29.7791 +0100",
"procRole" : "Unspecified",
"version" : 2,
"userID" : 501,
"deployVersion" : 210,
"modelCode" : "MacBookPro14,3",
"procStartAbsTime" : 91844701503187,
"coalitionID" : 1244,
"osVersion" : {
"train" : "macOS 12.0.1",
"build" : "21A559",
"releaseType" : "User"
},
"captureTime" : "2021-10-30 18:49:32.4572 +0100",
"incident" : "92A89610-D70A-4D93-A974-A9018BB5C72A",
"bug_type" : "309",
"pid" : 77765,
"procExitAbsTime" : 91847378271126,
"cpuType" : "X86-64",
"procName" : "Helper",
...
When I preview the file or open it in the Console app, a traditional crash report is automatically generated:
-------------------------------------
Translated Report (Full Report Below)
-------------------------------------
Process: Helper [77765]
Path: /Users/USER/Library/Application Support/Helper.app/Contents/MacOS/Helper
Identifier: com.distl.helper
Version: 3.0.0(66) beta (3.0.0.66)
Code Type: X86-64 (Native)
Parent Process: TestBead [77726]
Responsible: TestBead [77726]
User ID: 501
Date/Time: 2021-10-30 18:49:32.4572 +0100
OS Version: macOS 12.0.1 (21A559)
Report Version: 12
Bridge OS Version: 3.0 (14Y908)
Anonymous UUID: CC03C2EC-C1D4-4F6E-AA1F-6C4EC555D6B8
Time Awake Since Boot: 91000 seconds
System Integrity Protection: enabled
Crashed Thread: 1 Dispatch queue: com.apple.NSXPCConnection.user.anonymous.77726
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x00007f780071a000
Exception Codes: 0x0000000000000001, 0x00007f780071a000
Exception Note: EXC_CORPSE_NOTIFY
Termination Reason: Namespace SIGNAL, Code 11 Segmentation fault: 11
Terminating Process: exc handler [77765]
...
I have customer support and development tools that scan these crash report files automatically, and I'd like to find out if there's a way to automate the translation of the JSON data back into the traditional crash report format?
I'd like to do this to (a) avoid rewriting my crash report scanning tools (although that wouldn't be impossible), and (b) automatically translate these files into a human readable format, without resorting to opening the file in the Console app.
I've run into the same problem. I haven't tried it myself yet, but someone has already created an ips2crash command available at GitHub. As the name implies, it should convert an .ips file to the (now) legacy crash report format.
I need to create, among other items, an SQL instance on the google cloud platform. I am required to create the database using Jinja files.
My Yaml file
imports:
- path: companyInstance.jinja
resources:
# Creates a database instance
- name: dbinstance11
type: companyInstance.jinja
My companyInstance.jinja
resources:
- name: test
type: sqladmin.v1beta4.instance
properties:
zone: europe-west1
settings:
- tier: db-custom-4-15360
Currently, this should all work as far as I can tell but on a full deployment, I get an error message claiming that type is given an array type instead of object.
The full error is below:
ERROR: (gcloud.deployment-manager.deployments.create) Error in Operation [operation-169876402531-756fgfg342844d-6gd6ebb6-8bb655a]: errors:
- code: CONDITION_NOT_MET
location: /deployments/dep/resources/test->$.properties
message: |
error: instance type (array) does not match any allowed primitive type (allowed: ["object"]) level: "error"
schema: {"loadingURI":"#","pointer":"/schemas/Settings"}
instance: {"pointer":"/settings"}
domain: "validation"
keyword: "type"
found: "array"
expected: ["object"]
I am trying to clean-up the jmeter docker+ci pipeline of our functional tests. I see taurus has a clean way to run jmeter scripts in a container and it does the heavy lifting of downloading the version of jmeter I want + installing the plugins my scripts use - excellent.
Now I need to generate the reports in junit.xml so I could keep the reporting consistent. Up until now I was using a modified fork of https://github.com/tguzik/m2u to convert jtl reports to junit.xml
Appreciate any help with how I can get request, response (code & body) for all samples onto junit.xml (at least for the failed samples)?
I tried few variations of taurus yaml ...
reporting:
- module: console
- module: final_stats
summary: true
percentiles: true
test-duration: true
- module: junit-xml
filename: report/report.xml
data-source: sample-labels
reporting:
- module: console
- module: final_stats
summary: true
percentiles: true
test-duration: true
- module: passfail
- module: junit-xml
filename: report/report.xml
data-source: pass-fail
Also added certain passfail criteria variations on the passfail module. did not help
After fiddling with this for few hours, I believe there is no clean way to get anything meaningful onto the junit .xml report from the junit-xml module in taurus. It appears barebone. I also noticed that it could mess up the default jenkins junit plugin test result summary.
So I settled down with the following yaml setting and continued to use m2u.jar to convert the jtl to junit.xml
modules:
jmeter:
path: ~/.bzt/jmeter-taurus/bin/jmeter
download-link: https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-{version}.zip
version: 5.3
force-ctg: true
detect-plugins: true
plugins:
- jpgc-json=2.2
- jmeter-ftp
- jpgc-casutg
xml-jtl-flags:
xml: true
fieldNames: true
time: true
timestamp: true
latency: true
connectTime: false
success: true
label: true
code: true
message: true
threadName: true
dataType: false
encoding: false
assertions: true
subresults: true
responseData: false
samplerData: false
responseHeaders: false
requestHeaders: true
responseDataOnError: true
saveAssertionResultsFailureMessage: true
bytes: true
threadCounts: false
url: true
execution:
- write-xml-jtl: full
scenario:
script: v_jmxfilename
properties:
environment: v_env
reporting:
- module: console
- module: final_stats
summary: true
percentiles: true
test-duration: true
# - module: junit-xml
# filename: report/junit-report.xml
# data-source: sample-labels
As per JUnit-XML-Reporter documentation currently this is not possible:
This reporter provides test results in JUnit XML format parseable by Jenkins JUnit Plugin. Reporter has two options:
filename (full path to report file, optional. By default xunit.xml in artifacts dir)
data-source (which data source to use: sample-labels or pass-fail)
If sample-labels used as source data, report will contain urls with test errors. If pass-fail used as source data, report will contain Pass/Fail criteria information. Please note that you have to place pass-fail module in reporters list, before junit-xml module.
Taurus is not only for JMeter, it supports many more tools and not all of them provide possibility to store request and response data so the options I can think of are in:
Add a Listener to your Test Plan and choose what metrics you need to store into a separate file, the easiest one for using is Flexible File Writer
Use ShellExec Service to run your m2u.jar from Taurus config YAML
I'm using caffe to train a model. I am sure I have connected the data layer with the train.txt in the source of image_data_param.But when I try ./train.sh. It always prompt: can not find image.
Ubuntu 18.04 openCV3 python2
layer {
name: "data"
type: "ImageLabelmapData"
top: "data"
top: "label"
include {
phase: TRAIN
}
transform_param {
mirror: false
mean_value: 104.00699
mean_value: 116.66877
mean_value: 122.67892
}
image_data_param {
root_folder: "/home/yogang/Desktop/rawdata/train/"
source: "/home/yogang/Desktop/rawdata/train.txt"
batch_size: 1
shuffle: true
new_height:0
new_width: 0
}
}
I0828 19:29:13.834946 14079 layer_factory.hpp:77] Creating layer data
I0828 19:29:13.835011 14079 net.cpp:101] Creating Layer data
I0828 19:29:13.835031 14079 net.cpp:409] data -> data
I0828 19:29:13.835059 14079 net.cpp:409] data -> label
I0828 19:29:13.835124 14079 image_labelmap_data_layer.cpp:42] Opening file /home/yogang/Desktop/rawdata/train.txt
I0828 19:29:13.835505 14079 image_labelmap_data_layer.cpp:52] Shuffling data
I0828 19:29:13.835677 14079 image_labelmap_data_layer.cpp:57] A total of 242 images.
E0828 19:29:13.836748 14079 io.cpp:80] Could not open or find file /home/yogang/Desktop/rawdata/train//home/yogang/Desktop/rawdata/train/satellite144.jpg
E0828 19:29:13.836797 14079 io.cpp:80] Could not open or find file /home/yogang/Desktop/rawdata/train//home/yogang/Desktop/rawdata/train/400.jpg
F0828 19:29:13.836818 14079 image_labelmap_data_layer.cpp:86] Check failed: cv_img.data Could not load /home/yogang/Desktop/rawdata/train/satellite144.jpg
*** Check failure stack trace: ***
./train.sh: line 8: 14079 Aborted (core dumped) ./solve.py
i think image path is wrong
check your image path
I'm trying to use the rabbitmq_parameter ansible module to set a federation upstream set, while dynamically generating the set, with something like this:
- name: Set federation upstream set
rabbitmq_parameter:
component: federation-upstream-set
name: my-upstreams
vhost: my-vhost
value: "{{ my_upstream_set }}"
The variable my_upstream_set is defined in a separate host variable file, like so:
my_upstream_set: [{"upstream": "upstream1"}, {"upstream": "upstream2"}]
However, no matter how I generate the value argument, which must be json, (with or without quotes, with simple or double quotes, yaml or json formatted), I can't get this to work. I get either the task failing with "stderr: Error: JSON decoding error", or the following error:
failed: [myhost] => {"failed": true, "parsed": false}
invalid output was: Traceback (most recent call last):
File "<stdin>", line 1498, in <module>
File "<stdin>", line 142, in main
File "<stdin>", line 104, in set
File "<stdin>", line 88, in _exec
File "<stdin>", line 1351, in run_command
File "/usr/lib/python2.7/posixpath.py", line 261, in expanduser
if not path.startswith('~'):
AttributeError: 'list' object has no attribute 'startswith'
debug3: mux_client_read_packet: read header failed: Broken pipe
debug2: Received exit status from master 1
I've tried running the task with a hardcoded value (so, directly in the task file) and it works as expected, but I have no way of integrating variables into that. Any idea what I might be doing wrong here? Thanks!
my_upstream_set should be a JSON (string) in your case it is a list.