I was using xgboost version 0.6 when I pickled some pickle objects.
Now I upgraded to version 0.82 and when I'm trying to unpickle the old models I get:
AttributeError: 'XGBClassifier' object has no attribute 'kwargs'
I would really like to use these model without re training them, is there any way to open these pickles?
The new xgboost requires that objects will have a "kwargs" attribute, which old models do not have. One way to solve this is to downgrade to the old xgboost version, open them, add to each model the model.kwargs=None and then save them again, they should now work...
Another workaround is to hack the pickle file. You will load the pickle as a string, add the needed attribute and then load the pickle:
import re
xg_str = open('path_to_old_model.pkl').read()
kwargs_value= "kwargs'\np8\nNsS'"
new_xgboost = re.sub('colsample_bylevel', kwargs_value+"""colsample_bylevel""", xg_str)
new_model = pkl.loads(new_xgboost)
this adds "None" as the self.kwargs for your models. regex finds where the object attributes are declared, by searching for a known attribute in the model, "colsample_bylevel" and then adds before it another attribute.
To see how pickle encodes attributes you can create any class with some attributes and apply pkl.dumps to an instance. if it's a short class it will be pretty easy to read, that's how I got that "kwargs'\np8\nNsS'" means "kwargs=None".
Worked for me! I'm sure this can help with similar backwards compatibility issues with pickles, not neccesarily with this specific attribute.
Related
I've been trying to run code written in 2017, which uses tensorflow=1.3.0 and dm-sonnet=1.14. Many of the packages required have become unavailable and I was drowning in dependency issues so I decided to rewrite the code in a new version of tensorflow.
I have:
tensorflow=2.3.1
tensorflow-addons=0.11.2
tensorflow-estimator=2.3.0
tensorflow-probability=0.11.1
dm-sonnet=2.0.0
Many of the updates are just simple updating tf1 to tf2 but I have endless problems with dm-sonnet versions.
The code has:
class CosineWeights(snt.AbstractModule):
and I get:
AttributeError: module 'sonnet' has no attribute 'AbstractModule'
I suspect this is just a matter of updating the code of sonnet=1.14 to sonnet 2.0 but I can't find a "translation" for it, or how AbstractModule is used now. Which module comes instead, is it enough to just do snt.Module? I would greatly appreciate any help!
Quote From https://github.com/deepmind/sonnet
It fixed my problem. Hope this will help~
Building your own module
Sonnet strongly encourages users to subclass snt.Module to define their own modules. Let's start by creating a simple Linear layer called MyLinear:
class MyLinear(snt.Module):
def __init__(self, output_size, name=None):
super(MyLinear, self).__init__(name=name)
self.output_size = output_size
Code :
from gensim.models.word2vec import Word2Vec
w2v = Word2Vec()
training_data = w2v.generate_training_data(settings, corpus)
Error :
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-45-bae554564046> in <module>
1 w2v = Word2Vec()
2 # Numpy ndarray with one-hot representation for [target_word, context_words]
----> 3 training_data = w2v.generate_training_data(settings, corpus)
AttributeError: 'Word2Vec' object has no attribute 'generate_training_data'
I even tried importing gensim.models.word2vec and tried every possibility but couldn't get it done.
Can someone help me with it?
Thanks in advance !
Yes, the gensim Word2Vec class doesn't have that method – and as far as I know, it never has.
And from your example usage, I'm not even sure what it might purport to do: a Word2Vec model needs to be provided data in the right format – it doesn't "generate" it (even as a translation from some other corpus).
I suspect you are looking at docs or a code example from some other unrelated library.
For using gensim's Word2Vec, you should rely on the gensim documentation & examples. The class docs include some basic details of proper usage, and there's a Jupyter notebook word2vec.ipynb included with the library, in its docs/notebooks directory (and also viewable online).
I want to persist a trained model in CNTK and found the 'persist' functionality after some amount of searching. However, there seems to be some error in importing it.
from cntk import persist
This is throwing ImportError.
Am I doing something the wrong way? Or is this no longer supported? Is there an alternate way to persist a model?
persist is from an earlier beta. save_model is now a method of every CNTK function. So instead of doing save_model(z, filename) you do z.save_model(filename). Load_model works the same as before but you import it from cntk.ops.functions. For an example, see: https://github.com/Microsoft/CNTK/blob/v2.0.beta7.0/Tutorials/CNTK_203_Reinforcement_Learning_Basics.ipynb or https://github.com/Microsoft/CNTK/blob/v2.0.beta7.0/bindings/python/cntk/tests/persist_test.py
The functionality has moved to cntk functions. The new way is mynetwork.save_model(...) where mynetwork represents the root of your computation (typically the prediction). For loading the model you can just say mynetwork = C.load_model(...)
I'm working on a project using actionscript and Flex. For some reason I have a problem importing the com.adobe.serialization.json.JSON class.
When I'm working only with the FlexSDK files and try to use it I'm getting
the following error:
Error:(142, 70) [..]: Error code: 1120: Access of undefined property
JSON.
And of course IntelliJ marks this file and the import in red.
On the other hand when I import the corelib.swc that includes this file I get the following error:
Error:[..]: Can not resolve a multiname reference unambiguously. JSON
(from /Volumes/backup/FlexSDK/frameworks/libs/air/airglobal.swc(JSON,
Walker)) and com.adobe.serialization.json:JSON (from
/Volumes/backup/.../libs/corelib.swc(com.adobe.serialization.json:JSON))
are available.
What is going on here? How can I solve this?
JSON is a top level class available in all the scopes since FP11. Trying to import any class with name JSON will result in an error. If (for some reason) you really do not want to use the already available JSON class and instead import a custom one you'll have to rename it.
Using Intellij, the best you can do is use the JSON class from the current SDK that you have, it has the methods parse() and stringify(). those two methods do the same as the corelib methods for the json.
In case you wanted to use the com.adobe.serialization.json.JSON it will enter in conflict with the one declared in the SDK.
Hope the information is useful
A rails service I am currently working on requires that points are returned as a GeoJSON object within our json response. We are using rgeo and the mysql2spatial adapter to represent these points in our application and I would like to use the rgeo-geojson gem to handle the encoding if possible (we already use it to decode geojson on post).
I am currently overwriting as_json with the following code to achieve this:
def as_json(params)
l = {:lat_lng => ::RGeo::GeoJSON.encode(lat_lng)}
self.attributes.merge(l).as_json
end
However this is not optimal as the root (eg object: {}) is missing. Is there a function to easily include it? (a lot of our models have a lat_lng associated, so I'd rather not hard code it).
Any tips for a ruby/rails beginner would be greatly appreciated
For posterity, I fixed this in the "rgeo-activerecord" gem, version 0.3.4, after getting several reports on it. By default it renders spatial columns in WKT. To switch it to GeoJSON, set this:
RGeo::ActiveRecord::GeometryMixin.set_json_generator(:geojson)
The answer by NielsV will work sometimes but not every time. Specifically, it will work for geographic factories (i.e. geometry columns in PostGIS) but not for GEOS-backed factories.
You can specify it by including root with this line of code:
ActiveRecord::Base.include_root_in_json = true
I Hope this helps.
I solved this by extending the RGEO library with an as_json method for a Point, doing this it's no longer required to overwrite as_json in my own models. Thanks for your response though.
module RGeo
module Feature
module Point
def as_json(params)
::RGeo::GeoJSON.encode(self)
end
end
end
end