Mercurial Push Error on Google Code Value Error - mercurial

I am trying to learn to use mercurial by pushing onto Google code.
I have two .hgrc files: One file is located $PROJECT_DIR/.hg/.hrgc and $HOME/.hgrc. I have two separate files because I did not want to put the password out on the central repository.
Here is the content of $PROJECT_DIR/.hg/.hrgc:
[ui]
usermane=Venkat S. Rao <vrao423#gmail.com>
verbose=true
[paths]
default-push =https:vrao423:gc4yy3vB3mc4#//personal-site423.googlecode.com/hg/us
Here is the content of $HOME/.hgrc:
[ui]
username= Venkat Rao <vrao423#gmail.com>
verbose=True
[auth]
project.prefix=https://personal-site423.googlecode.com/hg/
password=###
username=vrao423
For username I have my Gmail id.
I can commit changes to my local repository, but when I try hg push I get this error.
** unknown exception encountered, details follow
** report bug details to http://mercurial.selenic.com/bts/
** or mercurial#selenic.com
** Mercurial Distributed SCM (version 1.4.3)
** Extensions loaded:
Traceback (most recent call last):
File "/usr/bin/hg", line 27, in
mercurial.dispatch.run()
File "/usr/lib/pymodules/python2.6/mercurial/dispatch.py", line 16, in run
sys.exit(dispatch(sys.argv[1:]))
File "/usr/lib/pymodules/python2.6/mercurial/dispatch.py", line 30, in dispatch
return _runcatch(u, args)
File "/usr/lib/pymodules/python2.6/mercurial/dispatch.py", line 46, in _runcatch
return _dispatch(ui, args)
File "/usr/lib/pymodules/python2.6/mercurial/dispatch.py", line 454, in _dispatch
return runcommand(lui, repo, cmd, fullargs, ui, options, d)
File "/usr/lib/pymodules/python2.6/mercurial/dispatch.py", line 324, in runcommand
ret = _runcommand(ui, options, cmd, d)
File "/usr/lib/pymodules/python2.6/mercurial/dispatch.py", line 505, in _runcommand
return checkargs()
File "/usr/lib/pymodules/python2.6/mercurial/dispatch.py", line 459, in checkargs
return cmdfunc()
File "/usr/lib/pymodules/python2.6/mercurial/dispatch.py", line 453, in
d = lambda: util.checksignature(func)(ui, *args, **cmdoptions)
File "/usr/lib/pymodules/python2.6/mercurial/util.py", line 386, in check
return func(*args, **kwargs)
File "/usr/lib/pymodules/python2.6/mercurial/commands.py", line 2345, in push
other = hg.repository(cmdutil.remoteui(repo, opts), dest)
File "/usr/lib/pymodules/python2.6/mercurial/hg.py", line 63, in repository
repo = _lookup(path).instance(ui, path, create)
File "/usr/lib/pymodules/python2.6/mercurial/httprepo.py", line 263, in instance
inst.between([(nullid, nullid)])
File "/usr/lib/pymodules/python2.6/mercurial/httprepo.py", line 184, in between
d = self.do_read("between", pairs=n)
File "/usr/lib/pymodules/python2.6/mercurial/httprepo.py", line 128, in do_read
fp = self.do_cmd(cmd, **args)
File "/usr/lib/pymodules/python2.6/mercurial/httprepo.py", line 80, in do_cmd
resp = self.urlopener.open(urllib2.Request(cu, data, headers))
File "/usr/lib/python2.6/urllib2.py", line 391, in open
response = self._open(req, data)
File "/usr/lib/python2.6/urllib2.py", line 409, in _open
'_open', req)
File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
result = func(*args)
File "/usr/lib/pymodules/python2.6/mercurial/url.py", line 455, in https_open
self.auth = self.pwmgr.readauthtoken(req.get_full_url())
File "/usr/lib/pymodules/python2.6/mercurial/url.py", line 141, in readauthtoken
group, setting = key.split('.', 1)
ValueError: need more than 1 value to unpack
Please help me. I have tried reading the hgrc man but that is just gibberish.
Thank You
Venkat

I'm a Mercurial developer. Please report problems with our man page on the mailinglist or on our bugtracker. I would love to hear from you so that we can make the man page better, so please write to us and tell us which part you found to be "gibberish".
In this particular case, the problem is that you need to write your auth section like this:
[auth]
project.prefix=https://personal-site423.googlecode.com/hg/
project.password=###
project.username=vrao423
where I would replace project with googlecode or something similar. We should of course report something sensible instead of a traceback and I can see that we already fixed this particular bug in Mercurial 1.5.

Related

How to resolve coreferences without Internet using AllenNLP and coref-spanbert-large?

A want to resolve coreferences without Internet using AllenNLP and coref-spanbert-large model.
I try to do it in the way that is describing here https://demo.allennlp.org/coreference-resolution
My code:
from allennlp.predictors.predictor import Predictor
import allennlp_models.tagging
predictor = Predictor.from_path(r"C:\Users\aap\Desktop\coref-spanbert-large-2021.03.10.tar.gz")
example = 'Paul Allen was born on January 21, 1953, in Seattle, Washington, to Kenneth Sam Allen and Edna Faye Allen.Allen attended Lakeside School, a private school in Seattle, where he befriended Bill Gates, two years younger, with whom he shared an enthusiasm for computers.'
pred = predictor.predict(document=example)
coref_res = predictor.coref_resolved(example)
print(pred)
print(coref_res)
When I have an access to internet the code works correctly.
But when I don't have an access to internet I get the following errors:
Traceback (most recent call last):
File "C:/Users/aap/Desktop/CoreNLP/Coref_AllenNLP.py", line 14, in <module>
predictor = Predictor.from_path(r"C:\Users\aap\Desktop\coref-spanbert-large-2021.03.10.tar.gz")
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\allennlp\predictors\predictor.py", line 361, in from_path
load_archive(archive_path, cuda_device=cuda_device, overrides=overrides),
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\allennlp\models\archival.py", line 206, in load_archive
config.duplicate(), serialization_dir
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\allennlp\models\archival.py", line 232, in _load_dataset_readers
dataset_reader_params, serialization_dir=serialization_dir
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\allennlp\common\from_params.py", line 604, in from_params
**extras,
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\allennlp\common\from_params.py", line 632, in from_params
kwargs = create_kwargs(constructor_to_inspect, cls, params, **extras)
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\allennlp\common\from_params.py", line 200, in create_kwargs
cls.__name__, param_name, annotation, param.default, params, **extras
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\allennlp\common\from_params.py", line 307, in pop_and_construct_arg
return construct_arg(class_name, name, popped_params, annotation, default, **extras)
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\allennlp\common\from_params.py", line 391, in construct_arg
**extras,
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\allennlp\common\from_params.py", line 341, in construct_arg
return annotation.from_params(params=popped_params, **subextras)
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\allennlp\common\from_params.py", line 604, in from_params
**extras,
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\allennlp\common\from_params.py", line 634, in from_params
return constructor_to_call(**kwargs) # type: ignore
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\allennlp\data\token_indexers\pretrained_transformer_mismatched_indexer.py", line 63, in __init__
**kwargs,
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\allennlp\data\token_indexers\pretrained_transformer_indexer.py", line 58, in __init__
model_name, tokenizer_kwargs=tokenizer_kwargs
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\allennlp\data\tokenizers\pretrained_transformer_tokenizer.py", line 71, in __init__
model_name, add_special_tokens=False, **tokenizer_kwargs
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\allennlp\common\cached_transformers.py", line 110, in get_tokenizer
**kwargs,
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\transformers\models\auto\tokenization_auto.py", line 362, in from_pretrained
config = AutoConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\transformers\models\auto\configuration_auto.py", line 368, in from_pretrained
config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\transformers\configuration_utils.py", line 424, in get_config_dict
use_auth_token=use_auth_token,
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\transformers\file_utils.py", line 1087, in cached_path
local_files_only=local_files_only,
File "C:\Users\aap\Desktop\CoreNLP\corenlp\lib\site-packages\transformers\file_utils.py", line 1268, in get_from_cache
"Connection error, and we cannot find the requested files in the cached path."
ValueError: Connection error, and we cannot find the requested files in the cached path. Please try again or make sure your Internet connection is on.
Process finished with exit code 1
Please, say me, what do I need to do my code works without Internet?
You will need a local copy of transformer model's configuration file and vocabulary so that the tokenizer and token indexer don't need to download those:
from transformers import AutoConfig, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(transformer_model_name)
config = AutoConfig.from_pretrained(transformer_model_name)
tokenizer.save_pretrained(local_config_path)
config.to_json_file(local_config_path + "/config.json")
You will then need to override the transformer model name in the configuration file to the local directory (local_config_path) where you saved these things:
predictor = Predictor.from_path(
r"C:\Users\aap\Desktop\coref-spanbert-large-2021.03.10.tar.gz",
overrides={
"dataset_reader.token_indexers.tokens.model_name": local_config_path,
"validation_dataset_reader.token_indexers.tokens.model_name": local_config_path,
"model.text_field_embedder.tokens.model_name": local_config_path,
},
)
I have run into similar problem when using structured-prediction-srl-bert without internet, and I saw in the logs 4 item for downloads:
dataset_reader.bert_model_name = bert-base-uncased, Downloading 4 files
model INFO vocabulary.py - Loading token dictionary from data/structured-prediction-srl-bert.2020.12.15/vocabulary. Downloading... 4x smaller files
Spacy models 'en_core_web_sm' not found
later on, [nltk_data] Error loading punkt: <urlopen error [Errno -3] Temporary failure in name resolution> [nltk_data] Error loading wordnet: <urlopen error [Errno -3] Temporary failure in name resolution>
I have solved it with these steps:
structured-prediction-srl-bert:
I have downloaded the structured-prediction-srl-bert.2020.12.15.tar.gz from the https://demo.allennlp.org/semantic-role-labeling (Model Card tab) -
https://storage.googleapis.com/allennlp-public-models/structured-prediction-srl-bert.2020.12.15.tar.gz
I have unzipped it into ./data/structured-prediction-srl-bert.2020.12.15
The code:
pip install allennlp==2.10.0 allennlp-models==2.10.0
from allennlp.predictors.predictor import Predictor
predictor = Predictor.from_path("./data/structured-prediction-srl-bert.2020.12.15/")
bert-base-uncased
I have created a folder ./data/bert-base-uncased and there I have downloaded these files from https://huggingface.co/bert-base-uncased/tree/main
config.json
tokenizer.json
tokenizer_config.json
vocab.txt
pytorch_model.bin
Aditionally, I had to change the "bert_model_name" from "bert-base-uncased" into a path "./data/bert-base-uncased", the earlier causes the download. This has to be done in the ./data/structured-prediction-srl-bert.2020.12.15/config.json , and there are two occurences.
python -m spacy download en_core_web_sm
python -c 'import nltk; nltk.download("punkt"); nltk.download("wordnet")'
After these steps the allennlp did not need internet anymore.

AWS Beanstalk integration with codecommit

I have existing python project on AWS Beanstalk I decided to integrate it with codecommit but I get following err when I enter Branch name.
Enter Branch Name
***** Must have at least one commit to create a new branch with CodeCommit *****
(default is "develop"): develop
Traceback (most recent call last):
File "/home/fartash/.local/bin/eb", line 11, in <module>
sys.exit(main())
File "/home/fartash/.local/lib/python3.5/site-packages/ebcli/core/ebcore.py", line 150, in main
app.run()
File "/home/fartash/.local/lib/python3.5/site-packages/cement/core/foundation.py", line 797, in run
return_val = self.controller._dispatch()
File "/home/fartash/.local/lib/python3.5/site-packages/cement/core/controller.py", line 472, in _dispatch
return func()
File "/home/fartash/.local/lib/python3.5/site-packages/cement/core/controller.py", line 478, in _dispatch
return func()
File "/home/fartash/.local/lib/python3.5/site-packages/ebcli/core/abstractcontroller.py", line 57, in default
self.do_command()
File "/home/fartash/.local/lib/python3.5/site-packages/ebcli/controllers/initialize.py", line 172, in do_command
branch = get_branch_interactive(repository)
File "/home/fartash/.local/lib/python3.5/site-packages/ebcli/controllers/initialize.py", line 575, in get_branch_interactive
source_control.setup_new_codecommit_branch(branch_name=branch_name)
File "/home/fartash/.local/lib/python3.5/site-packages/ebcli/objects/sourcecontrol.py", line 313, in setup_new_codecommit_branch
self.fetch_remote_branches(self.codecommit_remote_name)
File "/home/fartash/.local/lib/python3.5/site-packages/ebcli/objects/sourcecontrol.py", line 397, in fetch_remote_branches
stdout, stderr, exitcode = self._run_cmd(['git', 'fetch', self.get_codecommit_presigned_remote_url(),
File "/home/fartash/.local/lib/python3.5/site-packages/ebcli/objects/sourcecontrol.py", line 434, in get_codecommit_presigned_remote_url
signed_url = codecommit.create_signed_url(remote_url)
File "/home/fartash/.local/lib/python3.5/site-packages/ebcli/lib/codecommit.py", line 138, in create_signed_url
password = _sign_codecommit_url(aws.get_region_name(), remote_url)
File "/home/fartash/.local/lib/python3.5/site-packages/ebcli/lib/codecommit.py", line 130, in _sign_codecommit_url
signature = signer.signature(string_to_sign, request)
File "/home/fartash/anaconda3/lib/python3.5/site-packages/botocore/auth.py", line 305, in signature
k_date = self._sign(('AWS4' + key).encode('utf-8'),
TypeError: Can't convert 'NoneType' object to str implicitly

Cannot commit in mercurial repo

I have a Mercuirial (3.6.1) project. Every time I try to commit I get the next error.
(project)➜ project hg:(default) ✗ hg commit -m "TICK-190" subfolder/models.py
** unknown exception encountered, please report by visiting
** https://mercurial-scm.org/wiki/BugTracker
** Python 2.7.6 (default, Mar 22 2014, 22:59:56) [GCC 4.8.2]
** Mercurial Distributed SCM (version 3.6.1)
** Extensions loaded:
Traceback (most recent call last):
File "/home/django/project/bin/hg", line 43, in <module>
mercurial.dispatch.run()
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/dispatch.py", line 54, in run
sys.exit((dispatch(request(sys.argv[1:])) or 0) & 255)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/dispatch.py", line 116, in dispatch
ret = _runcatch(req)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/dispatch.py", line 187, in _runcatch
return _dispatch(req)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/dispatch.py", line 920, in _dispatch
cmdpats, cmdoptions)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/dispatch.py", line 679, in runcommand
ret = _runcommand(ui, options, cmd, d)
File "/home/django/plantbid/local/lib/python2.7/site-packages/mercurial/dispatch.py", line 1051, in _runcommand
return checkargs()
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/dispatch.py", line 1011, in checkargs
return cmdfunc()
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/dispatch.py", line 917, in <lambda>
d = lambda: util.checksignature(func)(ui, *args, **cmdoptions)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/util.py", line 801, in check
return func(*args, **kwargs)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/commands.py", line 1584, in commit
node = cmdutil.commit(ui, repo, commitfunc, pats, opts)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/cmdutil.py", line 2491, in commit
return commitfunc(ui, repo, message, matcher, opts)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/project.py", line 1578, in commitfunc
extra=extra)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/localrepo.py", line 64, in wrapper
return orig(repo.unfiltered(), *args, **kwargs)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/localrepo.py", line 1550, in commit
allowemptycommit = (wctx.branch() != wctx.p1().branch()
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/context.py", line 533, in branch
return encoding.tolocal(self._changeset[5].get("branch"))
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/util.py", line 534, in __get__
result = self.func(obj)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/context.py", line 502, in _changeset
return self._repo.changelog.read(self.rev())
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/changelog.py", line 338, in read
text = self.revision(node)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/revlog.py", line 1110, in revision
bins = self._chunks(chain, df=_df)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/revlog.py", line 1015, in _chunks
self._chunkraw(revs[0], revs[-1], df=df)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/revlog.py", line 990, in _chunkraw
return self._getchunk(start, length, df=df)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/revlog.py", line 981, in _getchunk
return self._loadchunk(offset, length, df=df)
File "/home/django/project/local/lib/python2.7/site-packages/mercurial/revlog.py", line 966, in _loadchunk
return util.buffer(d, offset - realoffset, length)
ValueError: size must be zero or positive
There's no much more detail I can explain. I was commiting my code without problems and suddenly this error started to show. I cannot commit any file. I can ask gor hg status with no problem.
Since this is a legacy project I've recently got, I want to make as little changes as possible.
UPDATE:
hg verify
repository uses revlog format 1
checking changesets
changelog#?: data length off by 1066185 bytes
changelog#?: rev 6801 points to nonexistent changeset -1
(expected 6801)
6802: unpacking changeset b087b38d690d: revlog decompress error: Error -5 while decompressing data: incomplete or truncated stream
checking manifests
manifest#?: data length off by 1894563 bytes
manifest#?: rev 6729 points to nonexistent changeset -1
manifest#?: 000000000000 not in changesets
manifest#?: rev 6730 points to unexpected changeset 6802
manifest#?: 9363dc75d5fb not in changesets
?: reading manifest delta 9363dc75d5fb: revlog decompress error: Error -5 while decompressing data: incomplete or truncated stream
crosschecking files in changesets and manifests
checking files
plantbid/templates/rfp_edit_nursery_responses3.html#?: rev 8 points to unexpected changeset 6802
(expected )
plantbid/templates/rfp_edit_nursery_responses3.html#?: 7e15c24e6859 not in manifests
plantrfp/views.py#4684: broken revlog! (index data/plantrfp/views.py.i is corrupted)
warning: orphan revlog 'data/plantrfp/views.py.i'
2266 files, 6803 changesets, 13181 total revisions
3 warnings encountered!
12 integrity errors encountered!
(first damaged changeset appears to be 4684)
You may have found a bug in Mercurial, but try updating to the latest Mercurial version (3.9) first before you file a bug report. https://www.mercurial-scm.org
Re:Update: That looks like your repository is faulty. Maybe your hard drive has errors? Is this the only copy of the repository, or is there another one somewhere else? You best bet is probably to try and find a non-broken copy.

Google Datastore SSL errors in Python from Google Compute Instances

I have a Python Django application running on a Google Compute instance. It is using gcloudoem to interface from Django to Google Datastore. gcloudoem uses the same underlying code to communicate with Datastore as gcloud-python 0.5.x
At what seems to be completely random times, I will get SSL errors happening when trying to talk to Datastore. There is no pattern in where in my application code these happen. It's just during a random call to Datastore. Here are the two flavours of errors:
ERROR:django.request:Internal Server Error: /complete/google-oauth2/
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 111, in get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/views/decorators/cache.py", line 52, in _wrapped_view_func
response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/views/decorators/csrf.py", line 57, in wrapped_view
return view_func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/social/apps/django_app/utils.py", line 51, in wrapper
return func(request, backend, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/social/apps/django_app/views.py", line 28, in complete
redirect_name=REDIRECT_FIELD_NAME, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/social/actions.py", line 43, in do_complete
user = backend.complete(user=user, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/social/backends/base.py", line 41, in complete
return self.auth_complete(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/social/utils.py", line 229, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/social/backends/oauth.py", line 387, in auth_complete
*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/social/utils.py", line 229, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/social/backends/oauth.py", line 396, in do_auth
return self.strategy.authenticate(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/social/strategies/django_strategy.py", line 96, in authenticate
return authenticate(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/contrib/auth/__init__.py", line 60, in authenticate
user = backend.authenticate(**credentials)
File "/usr/local/lib/python2.7/dist-packages/social/backends/base.py", line 82, in authenticate
return self.pipeline(pipeline, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/social/backends/base.py", line 85, in pipeline
out = self.run_pipeline(pipeline, pipeline_index, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/social/backends/base.py", line 112, in run_pipeline
result = func(*args, **out) or {}
File "/usr/local/lib/python2.7/dist-packages/social/pipeline/social_auth.py", line 20, in social_user
social = backend.strategy.storage.user.get_social_auth(provider, uid)
File "./social_gc/storage.py", line 105, in get_social_auth
return cls.objects.get(provider=provider, uid=uid)
File "/usr/local/lib/python2.7/dist-packages/gcloudoem/queryset/__init__.py", line 162, in get
num = len(clone)
File "/usr/local/lib/python2.7/dist-packages/gcloudoem/queryset/__init__.py", line 126, in __len__
self._fetch_all()
File "/usr/local/lib/python2.7/dist-packages/gcloudoem/queryset/__init__.py", line 370, in _fetch_all
self._result_cache = list(self.iterator())
File "/usr/local/lib/python2.7/dist-packages/gcloudoem/datastore/query.py", line 480, in __iter__
self.next_page()
File "/usr/local/lib/python2.7/dist-packages/gcloudoem/datastore/query.py", line 452, in next_page
transaction_id=transaction and transaction.id,
File "/usr/local/lib/python2.7/dist-packages/gcloudoem/datastore/connection.py", line 249, in run_query
response = self._rpc('runQuery', request, datastore_pb.RunQueryResponse)
File "/usr/local/lib/python2.7/dist-packages/gcloudoem/datastore/connection.py", line 159, in _rpc
data=request_pb.SerializeToString()
File "/usr/local/lib/python2.7/dist-packages/gcloudoem/datastore/connection.py", line 134, in _request
body=data
File "/usr/local/lib/python2.7/dist-packages/oauth2client/client.py", line 589, in new_request
redirections, connection_type)
File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1609, in request
(response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1351, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1307, in _conn_request
response = conn.getresponse()
File "/usr/lib/python2.7/httplib.py", line 1127, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 453, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 409, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "/usr/lib/python2.7/socket.py", line 480, in readline
data = self._sock.recv(self._rbufsize)
File "/usr/lib/python2.7/ssl.py", line 734, in recv
return self.read(buflen)
File "/usr/lib/python2.7/ssl.py", line 621, in read
v = self._sslobj.read(len or 1024)
SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1752)
Unfortunately, for the second, I don't have a full stacktrace handy:
[SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:1752)
These errors don't happen when I am using the GCD tool. Does anyone have any idea what is happening here? Is this some sort of networking problem?
I have also been receiving the [SSL: WRONG_VERSION_NUMBER] error when trying to use Datastore, however, I can repeat the error on demand. As James suggested, I get this error as soon as I introduce another thread querying Datastore. They are using completely separate application-level objects but I would imagine that as they are getting lower down in the gcloud library or lower down still there is some sort of object-sharing happening that is causing this problem.
UPDATE: I have since found the following very helpful thread (https://github.com/GoogleCloudPlatform/gcloud-python/issues/1214) which identifies an issue across the gcloud python apis due to a common dependency on the httplib2 library which turns out to not be thread-safe.
Somebody has written a wrapper for the gcloud suite that will use the requests library instead of httplib2 (gcloud requests) but it is built for Python 2.7. I didnt try to convert it for my Python3 project and instead used the very simple httplib2shim library to monkey-patch httplib2 with urllib3.
It was as simple as adding this :
import httplib2shim
httplib2shim.patch()
I'm now making calls from multiple threads without an issue.
: )
Two things come to mind which may be leading to this. Sorry this is not super specific; trying to help!
Threads - there are objects being shared across threads somehow which is causing the problem
Connections - There are too many connections being made, causing in failure (especially for the second error)

Django mysql when saving a object got warning

I was just writing a little app to store words in my mysql database using Django.I read data from a text file which is extremely well organised,like this:
The text file is like this:
DELUGE
DELUSION
DELVE
DEMAGOGUE
DEMANDING
DEMOLITION
DEMONSTRATE
DEMORALIZE
DEMOTIC
DEMUR
DENIGRATE
DENOUEMENT
DENOUNCE
DENT
DENUDE
DEPLETE
DEPLORE
DEPLOY
And then I read date from it using open('thefile').readlines like this:
for line in open('/home/jacos/sorted-gre.txt').readlines():
... if line:
... p = Word(word_spelling = line)
... p.save()
The word_spelling field is the primary key.
Then came this warning:
Traceback (most recent call last):
File "<console>", line 4, in <module>
File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 460, in save
self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 553, in save_base
result = manager._insert(values, return_id=update_pk, using=using)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/manager.py", line 195, in _insert
return insert_query(self.model, values, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 1436, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 791, in execute_sql
cursor = super(SQLInsertCompiler, self).execute_sql(None)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/util.py", line 34, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/mysql/base.py", line 86, in execute
return self.cursor.execute(query, args)
File "/usr/lib/pymodules/python2.7/MySQLdb/cursors.py", line 176, in execute
if not self._defer_warnings: self._warning_check()
File "/usr/lib/pymodules/python2.7/MySQLdb/cursors.py", line 92, in _warning_check
warn(w[-1], self.Warning, 3)
Data truncated for column 'word_spelling' at row 1
As a result, only part of these words were stored in mysql. I'd like to know why.
CharFields have a max_length attribute. What did you set when you generate database
for Word.object.get(pk=1).word_spelling?
nothing related with your warning but
it's recommended to close file or open it with a with statement.
with open('/home/jacos/sorted-gre.txt') as f:
for line in f.readlines():
if line:
p = Word(word_spelling = line)
p.save()