Unexpected and missing keys in state_dict when converting pytorch to onnx - deep-learning
When I convert a '.pth' model from PyTorch to ONNX, an error like Unexpected keys and missing keys occur.
This is my model:
1 import torch
2 import torch.onnx
3 from mmcv import runner
4 import torch.`enter code here`nn as nn
5 from mobilenet import MobileNet
6 # A model class instance (class not shown)
7 md=MobileNet(1,2)
8 model = md
9 device_ids = [0,2,6,7,8]
10 model = nn.DataParallel(model,device_ids)
11 #torch.backends.cudnn.benchmark = True
12 # Load the weights from a file (.pth usually)
13 runner.load_checkpoint(model,'../mmdetection- master/work_dmobile/faster_rcnn_r50_fpn_1x/epoch_60.pth')
14 #model = MMDataParallel(model, device_ids=[0])
15 #state_dict=torch.load('../mmdetection-master/r.pkl.json')
16 # Load the weights now into a model net architecture defined by our class
17 #model.load_state_dict(state_dict)
18 #model = runner.load_state_dict(state_dict)
19 model=runner.load_state_dict({k.replace('module.',' '):v for k,v in state_dict['state_dict'].items()})
20 # Create the right input shape (e.g. for an image)
21 dummy_input = torch.randn(1, 64, 512, 256)
22
23 torch.onnx.export(model, dummy_input, "onnx_model_name.onnx")
And this is the error:
unexpected key in source state_dict: backbone.stem.0.conv.weight, backbone.stem.0.bn.weight, backbone.stem.0.bn.bias, backbone.stem.0.bn.running_mean, backbone.stem.0.bn.running_var, backbone.stem.0.bn.num_batches_tracked, backbone.stem.1.depthwise.0.weight, backbone.stem.1.depthwise.1.weight, backbone.stem.1.depthwise.1.bias, backbone.stem.1.depthwise.1.running_mean, backbone.stem.1.depthwise.1.running_var, backbone.stem.1.depthwise.1.num_batches_tracked, backbone.stem.1.pointwise.0.weight, backbone.stem.1.pointwise.0.bias, backbone.stem.1.pointwise.1.weight, backbone.stem.1.pointwise.1.bias, backbone.stem.1.pointwise.1.running_mean, backbone.stem.1.pointwise.1.running_var, backbone.stem.1.pointwise.1.num_batches_tracked, backbone.conv1.0.depthwise.0.weight, backbone.conv1.0.depthwise.1.weight, backbone.conv1.0.depthwise.1.bias, backbone.conv1.0.depthwise.1.running_mean, backbone.conv1.0.depthwise.1.running_var, backbone.conv1.0.depthwise.1.num_batches_tracked, backbone.conv1.0.pointwise.0.weight, backbone.conv1.0.pointwise.0.bias, backbone.conv1.0.pointwise.1.weight, backbone.conv1.0.pointwise.1.bias, backbone.conv1.0.pointwise.1.running_mean, backbone.conv1.0.pointwise.1.running_var, backbone.conv1.0.pointwise.1.num_batches_tracked, backbone.conv1.1.depthwise.0.weight, backbone.conv1.1.depthwise.1.weight, backbone.conv1.1.depthwise.1.bias, backbone.conv1.1.depthwise.1.running_mean, backbone.conv1.1.depthwise.1.running_var, backbone.conv1.1.depthwise.1.num_batches_tracked, backbone.conv1.1.pointwise.0.weight, backbone.conv1.1.pointwise.0.bias, backbone.conv1.1.pointwise.1.weight, backbone.conv1.1.pointwise.1.bias, backbone.conv1.1.pointwise.1.running_mean, backbone.conv1.1.pointwise.1.running_var, backbone.conv1.1.pointwise.1.num_batches_tracked, backbone.conv2.0.depthwise.0.weight, backbone.conv2.0.depthwise.1.weight, backbone.conv2.0.depthwise.1.bias, backbone.conv2.0.depthwise.1.running_mean, backbone.conv2.0.depthwise.1.running_var, backbone.conv2.0.depthwise.1.num_batches_tracked, backbone.conv2.0.pointwise.0.weight, backbone.conv2.0.pointwise.0.bias, backbone.conv2.0.pointwise.1.weight, backbone.conv2.0.pointwise.1.bias, backbone.conv2.0.pointwise.1.running_mean, backbone.conv2.0.pointwise.1.running_var, backbone.conv2.0.pointwise.1.num_batches_tracked, backbone.conv2.1.depthwise.0.weight, backbone.conv2.1.depthwise.1.weight, backbone.conv2.1.depthwise.1.bias, backbone.conv2.1.depthwise.1.running_mean, backbone.conv2.1.depthwise.1.running_var, backbone.conv2.1.depthwise.1.num_batches_tracked, backbone.conv2.1.pointwise.0.weight, backbone.conv2.1.pointwise.0.bias, backbone.conv2.1.pointwise.1.weight, backbone.conv2.1.pointwise.1.bias, backbone.conv2.1.pointwise.1.running_mean, backbone.conv2.1.pointwise.1.running_var, backbone.conv2.1.pointwise.1.num_batches_tracked, backbone.conv3.0.depthwise.0.weight, backbone.conv3.0.depthwise.1.weight, backbone.conv3.0.depthwise.1.bias, backbone.conv3.0.depthwise.1.running_mean, backbone.conv3.0.depthwise.1.running_var, backbone.conv3.0.depthwise.1.num_batches_tracked, backbone.conv3.0.pointwise.0.weight, backbone.conv3.0.pointwise.0.bias, backbone.conv3.0.pointwise.1.weight, backbone.conv3.0.pointwise.1.bias, backbone.conv3.0.pointwise.1.running_mean, backbone.conv3.0.pointwise.1.running_var, backbone.conv3.0.pointwise.1.num_batches_tracked, backbone.conv3.1.depthwise.0.weight, backbone.conv3.1.depthwise.1.weight, backbone.conv3.1.depthwise.1.bias, backbone.conv3.1.depthwise.1.running_mean, backbone.conv3.1.depthwise.1.running_var, backbone.conv3.1.depthwise.1.num_batches_tracked, backbone.conv3.1.pointwise.0.weight, backbone.conv3.1.pointwise.0.bias, backbone.conv3.1.pointwise.1.weight, backbone.conv3.1.pointwise.1.bias, backbone.conv3.1.pointwise.1.running_mean, backbone.conv3.1.pointwise.1.running_var, backbone.conv3.1.pointwise.1.num_batches_tracked, backbone.conv3.2.depthwise.0.weight, backbone.conv3.2.depthwise.1.weight, backbone.conv3.2.depthwise.1.bias, backbone.conv3.2.depthwise.1.running_mean, backbone.conv3.2.depthwise.1.running_var, backbone.conv3.2.depthwise.1.num_batches_tracked, backbone.conv3.2.pointwise.0.weight, backbone.conv3.2.pointwise.0.bias, backbone.conv3.2.pointwise.1.weight, backbone.conv3.2.pointwise.1.bias, backbone.conv3.2.pointwise.1.running_mean, backbone.conv3.2.pointwise.1.running_var, backbone.conv3.2.pointwise.1.num_batches_tracked, backbone.conv3.3.depthwise.0.weight, backbone.conv3.3.depthwise.1.weight, backbone.conv3.3.depthwise.1.bias, backbone.conv3.3.depthwise.1.running_mean, backbone.conv3.3.depthwise.1.running_var, backbone.conv3.3.depthwise.1.num_batches_tracked, backbone.conv3.3.pointwise.0.weight, backbone.conv3.3.pointwise.0.bias, backbone.conv3.3.pointwise.1.weight, backbone.conv3.3.pointwise.1.bias, backbone.conv3.3.pointwise.1.running_mean, backbone.conv3.3.pointwise.1.running_var, backbone.conv3.3.pointwise.1.num_batches_tracked, backbone.conv3.4.depthwise.0.weight, backbone.conv3.4.depthwise.1.weight, backbone.conv3.4.depthwise.1.bias, backbone.conv3.4.depthwise.1.running_mean, backbone.conv3.4.depthwise.1.running_var, backbone.conv3.4.depthwise.1.num_batches_tracked, backbone.conv3.4.pointwise.0.weight, backbone.conv3.4.pointwise.0.bias, backbone.conv3.4.pointwise.1.weight, backbone.conv3.4.pointwise.1.bias, backbone.conv3.4.pointwise.1.running_mean, backbone.conv3.4.pointwise.1.running_var, backbone.conv3.4.pointwise.1.num_batches_tracked, backbone.conv3.5.depthwise.0.weight, backbone.conv3.5.depthwise.1.weight, backbone.conv3.5.depthwise.1.bias, backbone.conv3.5.depthwise.1.running_mean, backbone.conv3.5.depthwise.1.running_var, backbone.conv3.5.depthwise.1.num_batches_tracked, backbone.conv3.5.pointwise.0.weight, backbone.conv3.5.pointwise.0.bias, backbone.conv3.5.pointwise.1.weight, backbone.conv3.5.pointwise.1.bias, backbone.conv3.5.pointwise.1.running_mean, backbone.conv3.5.pointwise.1.running_var, backbone.conv3.5.pointwise.1.num_batches_tracked, backbone.conv4.0.depthwise.0.weight, backbone.conv4.0.depthwise.1.weight, backbone.conv4.0.depthwise.1.bias, backbone.conv4.0.depthwise.1.running_mean, backbone.conv4.0.depthwise.1.running_var, backbone.conv4.0.depthwise.1.num_batches_tracked, backbone.conv4.0.pointwise.0.weight, backbone.conv4.0.pointwise.0.bias, backbone.conv4.0.pointwise.1.weight, backbone.conv4.0.pointwise.1.bias, backbone.conv4.0.pointwise.1.running_mean, backbone.conv4.0.pointwise.1.running_var, backbone.conv4.0.pointwise.1.num_batches_tracked, backbone.conv4.1.depthwise.0.weight, backbone.conv4.1.depthwise.1.weight, backbone.conv4.1.depthwise.1.bias, backbone.conv4.1.depthwise.1.running_mean, backbone.conv4.1.depthwise.1.running_var, backbone.conv4.1.depthwise.1.num_batches_tracked, backbone.conv4.1.pointwise.0.weight, backbone.conv4.1.pointwise.0.bias, backbone.conv4.1.pointwise.1.weight, backbone.conv4.1.pointwise.1.bias, backbone.conv4.1.pointwise.1.running_mean, backbone.conv4.1.pointwise.1.running_var, backbone.conv4.1.pointwise.1.num_batches_tracked, neck.lateral_convs.0.conv.weight, neck.lateral_convs.0.conv.bias, neck.lateral_convs.1.conv.weight, neck.lateral_convs.1.conv.bias, neck.lateral_convs.2.conv.weight, neck.lateral_convs.2.conv.bias, neck.fpn_convs.0.conv.weight, neck.fpn_convs.0.conv.bias, neck.fpn_convs.1.conv.weight, neck.fpn_convs.1.conv.bias, neck.fpn_convs.2.conv.weight, neck.fpn_convs.2.conv.bias, rpn_head.rpn_conv.weight, rpn_head.rpn_conv.bias, rpn_head.rpn_cls.weight, rpn_head.rpn_cls.bias, rpn_head.rpn_reg.weight, rpn_head.rpn_reg.bias, bbox_head.fc_cls.weight, bbox_head.fc_cls.bias, bbox_head.fc_reg.weight, bbox_head.fc_reg.bias, bbox_head.shared_fcs.0.weight, bbox_head.shared_fcs.0.bias, bbox_head.shared_fcs.1.weight, bbox_head.shared_fcs.1.bias
missing keys in source state_dict: conv2.1.depthwise.1.weight, conv4.0.depthwise.0.weight, conv4.1.pointwise.1.weight, conv3.2.depthwise.0.weight, conv3.1.pointwise.0.weight, conv3.4.pointwise.1.bias, conv3.5.depthwise.1.bias, conv2.1.pointwise.1.weight, stem.1.pointwise.1.running_mean, conv3.3.pointwise.1.weight, conv3.3.depthwise.1.running_mean, conv3.1.depthwise.1.num_batches_tracked, conv3.0.depthwise.1.num_batches_tracked, conv2.1.depthwise.1.running_var, conv1.0.depthwise.1.weight, conv3.5.depthwise.1.running_var, stem.0.bn.bias, conv3.2.depthwise.1.num_batches_tracked, conv2.0.depthwise.0.weight, conv2.1.pointwise.0.bias, conv3.1.pointwise.1.bias, conv3.2.pointwise.1.bias, conv2.0.pointwise.1.num_batches_tracked, stem.1.pointwise.0.weight, conv2.0.depthwise.1.weight, stem.1.depthwise.0.weight, conv1.1.pointwise.1.weight, conv3.5.pointwise.0.weight, conv3.4.depthwise.1.running_var, conv1.0.pointwise.0.bias, conv3.3.depthwise.1.running_var, conv3.0.pointwise.1.weight, conv4.0.pointwise.1.num_batches_tracked, conv4.1.depthwise.1.running_var, stem.1.depthwise.1.running_var, conv3.0.pointwise.1.running_var, conv3.4.depthwise.0.weight, conv3.4.pointwise.1.num_batches_tracked, conv4.0.depthwise.1.num_batches_tracked, conv3.0.depthwise.1.weight, conv3.3.pointwise.0.bias, conv3.0.depthwise.1.running_mean, conv3.2.pointwise.1.running_mean, conv3.1.pointwise.0.bias, conv3.5.depthwise.1.num_batches_tracked, conv3.5.pointwise.1.running_mean, conv3.1.pointwise.1.running_var, conv1.0.depthwise.1.running_mean, stem.1.pointwise.1.bias, conv1.0.depthwise.0.weight, conv3.2.pointwise.0.weight, conv4.0.pointwise.1.running_mean, conv2.1.pointwise.1.running_mean, stem.1.pointwise.1.weight, conv4.1.depthwise.1.weight, conv4.0.pointwise.0.weight, conv1.1.depthwise.1.bias, conv3.2.pointwise.1.num_batches_tracked, conv4.1.depthwise.0.weight, conv3.4.depthwise.1.running_mean, conv1.0.depthwise.1.bias, conv2.0.pointwise.0.bias, conv3.4.depthwise.1.num_batches_tracked, conv4.1.pointwise.1.running_mean, conv2.1.depthwise.1.bias, conv3.2.depthwise.1.weight, conv2.0.pointwise.1.weight, conv1.0.pointwise.0.weight, conv3.1.depthwise.1.running_var, conv2.0.pointwise.1.bias, conv4.0.depthwise.1.bias, conv3.3.pointwise.1.running_var, conv3.4.pointwise.1.weight, conv4.0.pointwise.0.bias, conv3.4.depthwise.1.bias, conv4.1.depthwise.1.num_batches_tracked, conv2.0.pointwise.1.running_mean, conv1.1.depthwise.1.weight, conv2.0.pointwise.1.running_var, stem.1.depthwise.1.running_mean, conv3.4.pointwise.1.running_var, stem.1.depthwise.1.num_batches_tracked, conv3.3.depthwise.1.weight, stem.1.pointwise.1.running_var, conv4.1.depthwise.1.bias, conv3.0.pointwise.1.bias, conv2.0.depthwise.1.running_mean, conv1.1.pointwise.1.bias, conv4.1.pointwise.0.bias, conv3.2.pointwise.0.bias, conv1.1.pointwise.0.weight, conv1.0.pointwise.1.weight, conv1.0.pointwise.1.running_mean, stem.0.conv.weight, stem.1.depthwise.1.bias, conv3.3.depthwise.0.weight, conv1.1.depthwise.1.num_batches_tracked, conv3.3.pointwise.1.num_batches_tracked, conv3.2.pointwise.1.running_var, conv3.2.depthwise.1.running_mean, conv3.3.depthwise.1.bias, conv4.1.pointwise.1.num_batches_tracked, conv2.0.depthwise.1.num_batches_tracked, conv3.0.pointwise.0.bias, conv3.1.depthwise.1.running_mean, conv3.1.depthwise.1.weight, conv3.0.pointwise.1.num_batches_tracked, conv3.1.pointwise.1.weight, conv4.0.pointwise.1.bias, conv3.3.depthwise.1.num_batches_tracked, conv3.4.pointwise.0.weight, stem.1.pointwise.0.bias, conv3.0.depthwise.1.bias, conv1.1.pointwise.0.bias, conv4.0.pointwise.1.running_var, stem.0.bn.weight, conv1.0.pointwise.1.num_batches_tracked, conv2.1.depthwise.1.running_mean, conv4.1.depthwise.1.running_mean, conv1.1.pointwise.1.running_var, conv2.1.pointwise.1.num_batches_tracked, conv2.0.depthwise.1.running_var, conv3.5.depthwise.1.weight, conv3.0.depthwise.0.weight, conv4.0.depthwise.1.running_mean, stem.0.bn.num_batches_tracked, conv3.3.pointwise.1.running_mean, conv2.1.pointwise.1.running_var, conv3.0.pointwise.1.running_mean, conv1.1.depthwise.1.running_var, conv3.0.depthwise.1.running_var, conv1.0.depthwise.1.running_var, stem.1.pointwise.1.num_batches_tracked, conv4.0.pointwise.1.weight, conv1.1.pointwise.1.running_mean, conv2.1.depthwise.0.weight, conv1.0.depthwise.1.num_batches_tracked, conv1.0.pointwise.1.running_var, conv3.5.pointwise.1.weight, conv3.5.depthwise.1.running_mean, conv3.1.depthwise.1.bias, conv3.1.depthwise.0.weight, conv1.1.depthwise.1.running_mean, conv2.0.pointwise.0.weight, conv4.1.pointwise.1.bias, conv3.2.depthwise.1.running_var, conv3.5.pointwise.0.bias, conv3.4.depthwise.1.weight, conv3.2.depthwise.1.bias, stem.0.bn.running_mean, conv4.0.depthwise.1.running_var, conv1.1.depthwise.0.weight, stem.0.bn.running_var, conv4.1.pointwise.0.weight, conv2.1.pointwise.1.bias, conv3.4.pointwise.0.bias, conv1.0.pointwise.1.bias, conv3.5.pointwise.1.running_var, conv1.1.pointwise.1.num_batches_tracked, conv3.1.pointwise.1.running_mean, conv2.1.depthwise.1.num_batches_tracked, conv2.1.pointwise.0.weight, stem.1.depthwise.1.weight, conv3.5.pointwise.1.bias, conv3.5.pointwise.1.num_batches_tracked, conv3.1.pointwise.1.num_batches_tracked, conv3.2.pointwise.1.weight, conv3.5.depthwise.0.weight, conv3.3.pointwise.0.weight, conv2.0.depthwise.1.bias, conv3.0.pointwise.0.weight, conv3.3.pointwise.1.bias, conv3.4.pointwise.1.running_mean, conv4.0.depthwise.1.weight, conv4.1.pointwise.1.running_var
In line 19, try using model=runner.load_state_dict(..., strict=False).
Using the parameter strict=False tells the load_state_dict function that there might be missing keys in the checkpoint, which usually come from the BatchNorm layer as I see in this case.
Related
Load custom package model to get model vocabulary in AllenNLP python interface
I'm trying to get the vocabulary from some publicly-available pre-trained models (that aren't mine) using the python interface of AllenNLP, using self.vocab. However, I'm running into problems trying to load in the model. I'm looking to get the vocabulary from the dygiepp models, using the following code:
from allennlp.models.model import Model
scierc_model = Model.from_archive('https://s3-us-west-2.amazonaws.com/ai2-s2-research/dygiepp/master/scierc.tar.gz')
However, I get the following error:
---------------------------------------------------------------------------
ConfigurationError Traceback (most recent call last)
/tmp/local/63381207/ipykernel_7616/3549263982.py in <module>
----> 1 scierc_model = Model.from_archive('https://s3-us-west-2.amazonaws.com/ai2-s2-research/dygiepp/master/scierc.tar.gz')
~/anaconda3/envs/dygiepp/lib/python3.7/site-packages/allennlp/models/model.py in from_archive(cls, archive_file, vocab)
480 from allennlp.models.archival import load_archive # here to avoid circular imports
481
--> 482 model = load_archive(archive_file).model
483 if vocab:
484 model.vocab.extend_from_vocab(vocab)
~/anaconda3/envs/dygiepp/lib/python3.7/site-packages/allennlp/models/archival.py in load_archive(archive_file, cuda_device, overrides, weights_file)
231 # Instantiate model and dataset readers. Use a duplicate of the config, as it will get consumed.
232 dataset_reader, validation_dataset_reader = _load_dataset_readers(
--> 233 config.duplicate(), serialization_dir
234 )
235 model = _load_model(config.duplicate(), weights_path, serialization_dir, cuda_device)
~/anaconda3/envs/dygiepp/lib/python3.7/site-packages/allennlp/models/archival.py in _load_dataset_readers(config, serialization_dir)
267
268 dataset_reader = DatasetReader.from_params(
--> 269 dataset_reader_params, serialization_dir=serialization_dir
270 )
271 validation_dataset_reader = DatasetReader.from_params(
~/anaconda3/envs/dygiepp/lib/python3.7/site-packages/allennlp/common/from_params.py in from_params(cls, params, constructor_to_call, constructor_to_inspect, **extras)
586 "type",
587 choices=as_registrable.list_available(),
--> 588 default_to_first_choice=default_to_first_choice,
589 )
590 subclass, constructor_name = as_registrable.resolve_class_name(choice)
~/anaconda3/envs/dygiepp/lib/python3.7/site-packages/allennlp/common/params.py in pop_choice(self, key, choices, default_to_first_choice, allow_class_names)
322 """{"model": "my_module.models.MyModel"} to have it imported automatically."""
323 )
--> 324 raise ConfigurationError(message)
325 return value
326
ConfigurationError: dygie not in acceptable choices for dataset_reader.type: ['babi', 'conll2003', 'interleaving', 'multitask', 'multitask_shim', 'sequence_tagging', 'sharded', 'text_classification_json']. You should either use the --include-package flag to make sure the correct module is loaded, or use a fully qualified class name in your config file like {"model": "my_module.models.MyModel"} to have it imported automatically.
The error describes how to fix the error from the command line, but not in the python interface. I additionally tried adding the line import dygie to my code to import the missing package, but that didn't solve the problem.
Wondering if anyone knows how to get around this?
To run this model, you'll need to have the code from this repo: https://github.com/dwadden/dygiepp. In particular, you need to import the DyGIE dataset reader from here: https://github.com/dwadden/dygiepp/blob/master/dygie/data/dataset_readers/dygie.py#L29
Julia CSV.read not recognizing "select" keyword
I am reading in a space-delimited file using the CSV library in Julia. edgeList = CSV.read( joinpath(dataDirectory, "out.file"), types=[Int, Int], header=["node1", "node2"], skipto=3, select=[1,2] ) This yields the following error: MethodError: no method matching CSV.File(::String; types=DataType[Int64, Int64], header=["node1", "node2"], skipto=3, select=[1, 2]) Closest candidates are: CSV.File(::Any; header, normalizenames, datarow, skipto, footerskip, limit, transpose, comment, use_mmap, ignoreemptylines, missingstrings, missingstring, delim, ignorerepeated, quotechar, openquotechar, closequotechar, escapechar, dateformat, decimal, truestrings, falsestrings, type, types, typemap, categorical, pool, strict, silencewarnings, threaded, debug, parsingdebug, allowmissing) at /Users/n.jordanjameson/.julia/packages/CSV/4GOjG/src/CSV.jl:221 got unsupported keyword argument "select" I am using Julia v. 1.6.2. Here is the output versioninfo(): Julia Version 1.6.2 Commit 1b93d53fc4 (2021-07-14 15:36 UTC) Platform Info: OS: macOS (x86_64-apple-darwin18.7.0) CPU: Intel(R) Core(TM) i7-5650U CPU # 2.20GHz WORD_SIZE: 64 LIBM: libopenlibm LLVM: libLLVM-11.0.1 (ORCJIT, broadwell) The version of CSV is 0.10.4. The wiki for this version of CSV is here: https://csv.juliadata.org/stable/reading.html#CSV.read, and it has a select / drop entry. The file I am trying to read is from here: http://konect.cc/networks/moreno_crime/ (the file I'm using is called "out.moreno_crime_crime"). The first few lines are: % bip unweighted % 1476 829 551 1 1 1 2 1 3 1 4 2 5 2 6 2 7 2 8 2 9 2 10
I get a different error than you, can you restart Julia and make sure?
julia> CSV.read("/home/akako/Downloads/moreno_crime/out.moreno_crime_crime"; types=[Int, Int],
header=["node1", "node2"],
skipto=3,
select=[1,2]
)
ERROR: ArgumentError: provide a valid sink argument, like `using DataFrames; CSV.read(source, DataFrame)`
Stacktrace:
[1] read(source::String, sink::Nothing; copycols::Bool, kwargs::Base.Pairs{Symbol, Any, NTuple{4, Symbol}, NamedTuple{(:types, :header, :skipto, :select), Tuple{Vector{DataType}, Vector{String}, Int64, Vector{Int64}}}})
# CSV ~/.julia/packages/CSV/jFiCn/src/CSV.jl:89
[2] top-level scope
# REPL[8]:1
Stacktrace:
this error is telling you you can't CSV.read without a target sink, you might want to use CSV.File
julia> CSV.File("/home/akako/Downloads/moreno_crime/out.moreno_crime_crime"; types=[Int, Int],
header=["node1", "node2"],
skipto=3,
select=[1,2]
)
┌ Warning: thread = 1 warning: parsed expected 2 columns, but didn't reach end of line around data row: 1. Parsing extra columns and widening final columnset
└ # CSV ~/.julia/packages/CSV/jFiCn/src/file.jl:579
1476-element CSV.File:
CSV.Row: (node1 = 1, node2 = 1, Column3 = missing)
CSV.Row: (node1 = 1, node2 = 2, Column3 = missing)
CSV.Row: (node1 = 1, node2 = 3, Column3 = missing)
CSV.Row: (node1 = 1, node2 = 4, Column3 = missing)
How to use HuggingFace nlp library's GLUE for CoLA
I've been trying to use the HuggingFace nlp library's GLUE metric to check whether a given sentence is a grammatical English sentence. But I'm getting an error and is stuck without being able to proceed.
What I've tried so far;
reference and prediction are 2 text sentences
!pip install transformers
from transformers import BertTokenizer
tokenizer = BertTokenizer.from_pretrained('bert-large-uncased')
reference="Security has been beefed across the country as a 2 day nation wide curfew came into effect."
prediction="Security has been tightened across the country as a 2-day nationwide curfew came into effect."
import nlp
glue_metric = nlp.load_metric('glue',name="cola")
#Using BertTokenizer
encoded_reference=tokenizer.encode(reference, add_special_tokens=False)
encoded_prediction=tokenizer.encode(prediction, add_special_tokens=False)
glue_score = glue_metric.compute(encoded_prediction, encoded_reference)
Error I'm getting;
ValueError Traceback (most recent call last)
<ipython-input-9-4c3a3ce7b583> in <module>()
----> 1 glue_score = glue_metric.compute(encoded_prediction, encoded_reference)
6 frames
/usr/local/lib/python3.6/dist-packages/nlp/metric.py in compute(self, predictions, references, timeout, **metrics_kwargs)
198 predictions = self.data["predictions"]
199 references = self.data["references"]
--> 200 output = self._compute(predictions=predictions, references=references, **metrics_kwargs)
201 return output
202
/usr/local/lib/python3.6/dist-packages/nlp/metrics/glue/27b1bc63e520833054bd0d7a8d0bc7f6aab84cc9eed1b576e98c806f9466d302/glue.py in _compute(self, predictions, references)
101 return pearson_and_spearman(predictions, references)
102 elif self.config_name in ["mrpc", "qqp"]:
--> 103 return acc_and_f1(predictions, references)
104 elif self.config_name in ["sst2", "mnli", "mnli_mismatched", "mnli_matched", "qnli", "rte", "wnli", "hans"]:
105 return {"accuracy": simple_accuracy(predictions, references)}
/usr/local/lib/python3.6/dist-packages/nlp/metrics/glue/27b1bc63e520833054bd0d7a8d0bc7f6aab84cc9eed1b576e98c806f9466d302/glue.py in acc_and_f1(preds, labels)
60 def acc_and_f1(preds, labels):
61 acc = simple_accuracy(preds, labels)
---> 62 f1 = f1_score(y_true=labels, y_pred=preds)
63 return {
64 "accuracy": acc,
/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py in f1_score(y_true, y_pred, labels, pos_label, average, sample_weight, zero_division)
1097 pos_label=pos_label, average=average,
1098 sample_weight=sample_weight,
-> 1099 zero_division=zero_division)
1100
1101
/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py in fbeta_score(y_true, y_pred, beta, labels, pos_label, average, sample_weight, zero_division)
1224 warn_for=('f-score',),
1225 sample_weight=sample_weight,
-> 1226 zero_division=zero_division)
1227 return f
1228
/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py in precision_recall_fscore_support(y_true, y_pred, beta, labels, pos_label, average, warn_for, sample_weight, zero_division)
1482 raise ValueError("beta should be >=0 in the F-beta score")
1483 labels = _check_set_wise_labels(y_true, y_pred, average, labels,
-> 1484 pos_label)
1485
1486 # Calculate tp_sum, pred_sum, true_sum ###
/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py in _check_set_wise_labels(y_true, y_pred, average, labels, pos_label)
1314 raise ValueError("Target is %s but average='binary'. Please "
1315 "choose another average setting, one of %r."
-> 1316 % (y_type, average_options))
1317 elif pos_label not in (None, 1):
1318 warnings.warn("Note that pos_label (set to %r) is ignored when "
ValueError: Target is multiclass but average='binary'. Please choose another average setting, one of [None, 'micro', 'macro', 'weighted'].
However, I'm able to get results (pearson and spearmanr) for 'stsb' with the same workaround as given above.
Some help and a workaround for(cola) this is really appreciated. Thank you.
In general, if you are seeing this error with HuggingFace, you are trying to use the f-score as a metric on a text classification problem with more than 2 classes. Pick a different metric, like "accuracy". For this specific question: Despite what you entered, it is trying to compute the f-score. From the example notebook, you should set the metric name as: metric_name = "pearson" if task == "stsb" else "matthews_correlation" if task == "cola" else "accuracy"
what type of model can i use to train this data
I have downloaded and labeled data from http://archive.ics.uci.edu/ml/datasets/pamap2+physical+activity+monitoring my task is to gain an insight into the data from what is given, I have round 34 attributes in a data frame(all clean no nan values) and want to train a model based on one target attribute 'heart_rate' given the rest of the attributes(all are numbers of a participant performing various activities ) I wanted to use Linear regression model but can not use my dataframe for some reason, however, I do not mind starting from 0 if you think I am doing it wrong my DataFrame columns: > Index(['timestamp', 'activity_ID', 'heart_rate', 'IMU_hand_temp', > 'hand_acceleration_16_1', 'hand_acceleration_16_2', > 'hand_acceleration_16_3', 'hand_gyroscope_rad_7', > 'hand_gyroscope_rad_8', 'hand_gyroscope_rad_9', > 'hand_magnetometer_μT_10', 'hand_magnetometer_μT_11', > 'hand_magnetometer_μT_12', 'IMU_chest_temp', 'chest_acceleration_16_1', > 'chest_acceleration_16_2', 'chest_acceleration_16_3', > 'chest_gyroscope_rad_7', 'chest_gyroscope_rad_8', > 'chest_gyroscope_rad_9', 'chest_magnetometer_μT_10', > 'chest_magnetometer_μT_11', 'chest_magnetometer_μT_12', > 'IMU_ankle_temp', 'ankle_acceleration_16_1', 'ankle_acceleration_16_2', > 'ankle_acceleration_16_3', 'ankle_gyroscope_rad_7', > 'ankle_gyroscope_rad_8', 'ankle_gyroscope_rad_9', > 'ankle_magnetometer_μT_10', 'ankle_magnetometer_μT_11', > 'ankle_magnetometer_μT_12', 'Intensity'], > dtype='object') first 5 rows: timestamp activity_ID heart_rate IMU_hand_temp hand_acceleration_16_1 hand_acceleration_16_2 hand_acceleration_16_3 hand_gyroscope_rad_7 hand_gyroscope_rad_8 hand_gyroscope_rad_9 ... ankle_acceleration_16_1 ankle_acceleration_16_2 ankle_acceleration_16_3 ankle_gyroscope_rad_7 ankle_gyroscope_rad_8 ankle_gyroscope_rad_9 ankle_magnetometer_μT_10 ankle_magnetometer_μT_11 ankle_magnetometer_μT_12 Intensity 2928 37.66 lying 100.0 30.375 2.21530 8.27915 5.58753 -0.004750 0.037579 -0.011145 ... 9.73855 -1.84761 0.095156 0.002908 -0.027714 0.001752 -61.1081 -36.8636 -58.3696 low 2929 37.67 lying 100.0 30.375 2.29196 7.67288 5.74467 -0.171710 0.025479 -0.009538 ... 9.69762 -1.88438 -0.020804 0.020882 0.000945 0.006007 -60.8916 -36.3197 -58.3656 low 2930 37.68 lying 100.0 30.375 2.29090 7.14240 5.82342 -0.238241 0.011214 0.000831 ... 9.69633 -1.92203 -0.059173 -0.035392 -0.052422 -0.004882 -60.3407 -35.7842 -58.6119 low 2931 37.69 lying 100.0 30.375 2.21800 7.14365 5.89930 -0.192912 0.019053 0.013374 ... 9.66370 -1.84714 0.094385 -0.032514 -0.018844 0.026950 -60.7646 -37.1028 -57.8799 low 2932 37.70 lying 100.0 30.375 2.30106 7.25857 6.09259 -0.069961 -0.018328 0.004582 ... 9.77578 -1.88582 0.095775 0.001351 -0.048878 -0.006328 -60.2040 -37.1225 -57.8847 low if you check the timestamp attribute you will see that the data acquired is in milliseconds so it might be a good idea to use the data from this dataframe as in every 2-5 seconds and train the model also as an option, I want to use as one of these models for this task Linear,polynomial, multiple linear, agglomerative clustering and kmeans clustering. my code: target = subject1.DataFrame(data.target, columns=["heart_rate"]) X = df y = target[“heart_rate”] lm = linear_model.LinearRegression() model = lm.fit(X,y) predictions = lm.predict(X) print(predictions)[0:5] Error: AttributeError Traceback (most recent call last) <ipython-input-93-b0c3faad3a98> in <module>() 3 #heart_rate 4 # Put the target (housing value -- MEDV) in another DataFrame ----> 5 target = subject1.DataFrame(data.target, columns=["heart_rate"]) c:\python36\lib\site-packages\pandas\core\generic.py in __getattr__(self, name) 5177 if self._info_axis._can_hold_identifiers_and_holds_name(name): 5178 return self[name] -> 5179 return object.__getattribute__(self, name) 5180 5181 def __setattr__(self, name, value): AttributeError: 'DataFrame' object has no attribute 'DataFrame' for fixing the error I have used: subject1.columns = subject1.columns.str.strip() but still did not work Thank you, sorry if I was not precise enough.
Try this:
X = df.drop("heart_rate", axis=1)
y = df[[“heart_rate”]]
X=X.apply(zscore)
test_size=0.30
seed=7
X_train, X_test, y_train, y_test=train_test_split(X, y, test_size=test_size, random_state=seed)
lm = linear_model.LinearRegression()
model = lm.fit(X,y)
predictions = lm.predict(X)
print(predictions)[0:5]
Hyperparameter tuning using tensorboard.plugins.hparams api with custom loss function
I am building a neural network with my own custom loss function (pretty long and complicated). My network is unsupervised so my input and expected output are identical and also at the moment I am using one single input (just trying to optimize the loss for a single input).
I am trying to use tensorboard.plugins.hparams api for hyperparameter tuning and don't know how to incorporate my custom loss function there. I'm trying to follow the code suggested on the Tensorflow 2.0 website.
This is what the website suggests:
HP_NUM_UNITS = hp.HParam('num_units', hp.Discrete([16, 32]))
HP_DROPOUT = hp.HParam('dropout', hp.RealInterval(0.1, 0.2))
HP_OPTIMIZER = hp.HParam('optimizer', hp.Discrete(['adam', 'sgd']))
METRIC_ACCURACY = 'accuracy'
with tf.summary.create_file_writer('logs/hparam_tuning').as_default():
hp.hparams_config(
hparams=[HP_NUM_UNITS, HP_DROPOUT, HP_OPTIMIZER],
metrics=[hp.Metric(METRIC_ACCURACY, display_name='Accuracy')],
)
I need to change that as I don't want to use the dropout layer, so I can just delete that. In terms of the METRIC_ACCURACY, I don't want to use accuracy as that has no use in my model but rather use my custom loss function. If I were to do the regular fit model it would look like this:
model.compile(optimizer=adam,loss=dl_tf_loss, metrics=[dl_tf_loss])
So I tried to change the suggested code into the following code but I get an error and am wondering how I should change it so that it suits my needs. Here is what I tried:
HP_NUM_UNITS = hp.HParam('num_units', hp.Discrete([16, 32]))
HP_OPTIMIZER = hp.HParam('optimizer', hp.Discrete(['adam', 'sgd']))
#METRIC_LOSS = dl_tf_loss
with tf.summary.create_file_writer('logs/hparam_tuning').as_default():
hp.hparams_config(hparams=[HP_NUM_UNITS, HP_OPTIMIZER],metrics=
[hp.Metric(dl_tf_loss, display_name='Loss')])
It gives me the following error:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-26-27d079c6be49> in <module>()
5
6 with tf.summary.create_file_writer('logs/hparam_tuning').as_default():
----> 7 hp.hparams_config(hparams=[HP_NUM_UNITS, HP_OPTIMIZER],metrics=[hp.Metric(dl_tf_loss, display_name='Loss')])
8
3 frames
/usr/local/lib/python3.6/dist-packages/tensorboard/plugins/hparams/summary_v2.py in hparams_config(hparams, metrics, time_created_secs)
127 hparams=hparams,
128 metrics=metrics,
--> 129 time_created_secs=time_created_secs,
130 )
131 return _write_summary("hparams_config", pb)
/usr/local/lib/python3.6/dist-packages/tensorboard/plugins/hparams/summary_v2.py in hparams_config_pb(hparams, metrics, time_created_secs)
161 domain.update_hparam_info(info)
162 hparam_infos.append(info)
--> 163 metric_infos = [metric.as_proto() for metric in metrics]
164 experiment = api_pb2.Experiment(
165 hparam_infos=hparam_infos,
/usr/local/lib/python3.6/dist-packages/tensorboard/plugins/hparams/summary_v2.py in <listcomp>(.0)
161 domain.update_hparam_info(info)
162 hparam_infos.append(info)
--> 163 metric_infos = [metric.as_proto() for metric in metrics]
164 experiment = api_pb2.Experiment(
165 hparam_infos=hparam_infos,
/usr/local/lib/python3.6/dist-packages/tensorboard/plugins/hparams/summary_v2.py in as_proto(self)
532 name=api_pb2.MetricName(
533 group=self._group,
--> 534 tag=self._tag,
535 ),
536 display_name=self._display_name,
TypeError: <tensorflow.python.eager.def_function.Function object at 0x7f9f3a78e5c0> has type Function, but expected one of: bytes, unicode
I also tried running the following code:
with tf.summary.create_file_writer('logs/hparam_tuning').as_default():
hp.hparams_config(hparams=[HP_NUM_UNITS, HP_OPTIMIZER],metrics=
[dl_tf_loss])
but got the following error:
AttributeError Traceback (most recent call last)
<ipython-input-28-6778bdf7f1b1> in <module>()
8
9 with tf.summary.create_file_writer('logs/hparam_tuning').as_default():
---> 10 hp.hparams_config(hparams=[HP_NUM_UNITS, HP_OPTIMIZER],metrics=[dl_tf_loss])
2 frames
/usr/local/lib/python3.6/dist-packages/tensorboard/plugins/hparams/summary_v2.py in <listcomp>(.0)
161 domain.update_hparam_info(info)
162 hparam_infos.append(info)
--> 163 metric_infos = [metric.as_proto() for metric in metrics]
164 experiment = api_pb2.Experiment(
165 hparam_infos=hparam_infos,
AttributeError: 'Function' object has no attribute 'as_proto'
Would greatly appreciate any help.
Thanks in advance!
I figured it out. The original METRIC_ACCURACY that I changed to METRIC_LOSS is apparently just the name, I needed to write 'tf_dl_loss' as a string and not as the function. In the proceeding parts of the tuning, I needed to anyway write my fit command, there I inserted the actual loss function as I showed in my example of the regular fit function. Highly recommend this as a way of tuning the hyperparameters.
You might be interested by this demo. Compiling the model with dl_tf_loss in the metric will waste time. It is possible to let hp.Metric know about other recorded summaries in different directories using the group argument.