I've written a small web-app which pulls data from a simple mysql DB and displays it via Python and Flask. The text fields in the database are encoded as utf8_general_ci, and some contain special characters - for example, 'Zürich'.
As Flask/Jinja2 like to work on unicode, after the query the strings are converted to unicode to be passed into the template. What's weird is I need a different approach to convert to unicode, depending on whether the code is running locally (Mac laptop) or deployed on GAE.
This works when running locally:
return [[unicode(c, encoding='utf-8') if isinstance(c, basestring) else c for c in b] for b in l]
This works when deployed on GAE:
return [[unicode(c, encoding='latin-1') if isinstance(c, basestring) else c for c in b] for b in l]
.
If I run the GAE version locally, 'Zürich' is displayed as 'Zürich'. Vice versa, I get an UnicodeDecode error.
As far as I can tell, the two databases are identical - the online version is a straight dump of the local version.
ü is the Mojibake for ü.
See "Mojibake" in here for a discussion of the likely causes.
See here for Python notes on what to do in the source code. I don't know about Jinja specifics.
Getting this very strange error when I am trying to load my ipython notebook. Never had it before, and I cannot to my recollection, remember having done anything silly with ipython:
Unreadable Notebook: /path/to/notebooks/results.ipynb NotJSONError('Notebook does not appear to be JSON: u\'{\\n "cells": [\\n {\\n "cell_type": "...',)
which is followed by
400 GET /api/contents/results.ipynb?type=notebook&_=1440010858974 (127.0.0.1) 36.17ms referer=http://localhost:8888/notebooks/results.ipynb
Save yourself a headache. Open your .ipynb in any online JSON validator and it will tell you which lines have issues. I used this one.
In my case, I am using GitHub to save and share my ipython files with my teammate. When there is a conflict in the code, I had to delete those lines indicating the changes in the conflicting code such as:
>>>>>>>>head
=============
and It works for me.
This happened to me as well. I opened my data.ipynb file using notepad and found out it was blank.
I managed to recover my file by going into the hidden ipynb_checkpoints folder and copying data_checkpoint.ipynb out into my working directory.
In my Mac OS terminal
cd .ipynb_checkpoints
cp data-checkpoint.ipynb \..
Thankfully the codes were preserved. Hope this helps!
I just had the same issue after upgrading from IPython 0.13 (ish) to Jupyter 4.
The problem in my case were a few rogue trailing commas in the JSON, for example the comma following "outputs" in:
...
"language": "python",
"metadata": {},
"outputs": [],
},
After removing the commas, Jupyter/IPython could again read the notebook (and upgraded it to version 4). I hope this helps.
The easiest way to recover corrupted Jupyter notebook files, whether it contains text or not (size = 0KB), is to go to the project folder and display the hidden files. Once the hidden files are displayed, you will see a folder named '.ipynb_checkpoints'. Simply open this folder and take the file you want!
Visual studio code procedure
This is my procedure that usually avoids me groping in the dark.
I installed a json parser validator like this one.
Open the file and save a copy as .json file.
Open the json and look for the errors.
Save it back to the .ipynb extension.
Usually, I manage to fix the errors quickly.
Jupyter autosaves in a specific way. It means You have accidentally closed the notebook before properly saving it.
You need to look for three things -
Search for <<<<<<< and delete those lines.
Search for ====== and replace those lines with ,.
Search for >>>>>>> and delete those lines.
It will work fine after this.
this can be changed to reformat your ipynb file to readable in jupyter notebook. check your other ipynb files(open in notepad) which are working fine with your jupyter notebook, check and compare at the end of the files in notepad. there you can reformat the file which is not working.
I had this issue from accidentally saving as .txt from github and solved by deleting .txt (leaving .ipynb instead of .ipynb.txt when downloading)
Yes, the best solution for me was I saved my notebook in HTML format, then opened it in Notepad ++ , delete the long repeated lines of output which were causing my notebook to grow to 45MB, once that cleared, Saved the file back into (.ipynb) format , and was able to opened it with no JSON error.
Hope that worked for others as well!
Got this error after conflicts while pushing my code to Github. The code present on the repo was old, and my changes were stashed. Notebook wasn't opening in either Jupyter and github repo. Following above comments, I searched for the part in my code which was giving JSON error,i.e. '<<<<<<<<<<<', '=======' and '>>>>>>>>>>' characters using an online json parser. Then I opened my .ipynb notebook in notepad++ and manually replaced these characters with blank string ''. After this, the notebook opened on my local Jupyter, and I also pushed the changes to Github.
I have changed by ipynb file encoding from UTF-8-BOM to UTF-8, and then it worked.
My native language is not English, but because this problem helped me a part, I came to feedback my solution.
The following is translated with translation software:
Fundamentally, the file format is messed up due to wrong closing. When opening, the correctness of the json format will be checked first, and an error will be returned if it is found to be wrong.
The mess in my file format is not <<<<< or ====== but the lack of commas.
Either way, it's best to use a piece of software to detect errors in the json syntax, and then manually fix it yourself.
The json website detection provided by the highest praise is available, but the detection errors are not complete, and may need to be detected-modified-detected-modified.
Also use vscode to open the file, vscode will prompt the location of the json syntax error, which is also incomplete and needs to be checked and modified multiple times.
The error location provided is more difficult to find. I use nodepad++, and the lower right corner can display how many characters are selected (standard, including line breaks). Then select from the first character until the destination position.
Although it's a bit stupid, the main reason is that I didn't find the relevant positioning method.
Clear all outputs.
Then copy the notebook.
If you use Jupyter-Notebook in VS code, just save it in VS code, close the file and try to open it again by accessing the browser.
on ubuntu 20.04, I have file String.ipynb. I had same problem because I coded ơ [ echo 'hello' >> String.ipynb ]. deleting 'hello' in String.ipynb -> I could open my notebook like normal.
how did I delete? [ nano String.ipynb ] * move to last line (hello) * -> delete it.
I hope my answer help you :D
you will see this error may be because, you were getting merge conflict in .ipynb file. because of that git adds >>>>>>>> HEAD thing in .ipynb file which makes is unreadable.
To overcome this issue open .ipynb file in vim editor and then remove the incoming changes or your changes as per your use case.
vim <your-.ipynb-file-path>
To remove incoming changes remove content between these lines<<<<<<<<<< HEAD ==============. Note:- remove this line as well >>>>>>>>>>>> this line.
to remove your changes remove content between these lines ============== >>>>>>>>>>>. Note:- remove this line as well a <<<<<<<<<< HEAD
I had the same issue after git merge while using VS Code and Jupyter extension.
VS Code would not open the notebook after the merge conflicts were highlighted in the notebook JSON by git (e.g. <<<<<). One way around it was to highlight the changes and accept one by one using the file viewer in the VSCode git interface.
Alternative that worked for me was to rename the file to .json so that it would open and then search for each instance of <<<<< and accept the incoming change.
The database is based on MSSql and Laravel is successfully talking to it. Database is base on latin2 encoding so my Laravel app has to be iso-8859-1 istead UTF-8. Seems like entire laravel is shouting UTF8 though. I can't find anything in config files.
I have changed database encoding in connection string to be lating1 but that still doesn't' change anything.
When an entry is made form VB desktop application (my colleague is working on that) and he saves a £ symbol to the db, it goes there fine, although when I try to read that it gives me � symbol.
On the other hand if from Form::open tag I create an entry in the database the £ gets prefixed with something weird!
Also I have added <meta charset="iso-8859-1"> to the html tag of the website but when i go in firefox to see page info still tells me that the website is in UTF8. I tried changing some of the files to the western encoding (Sublime save as ->with formating) like index.php, routes.php but the website still shows as UTF8...
Any help/ advice will be much appreciated.
To solve this I have done the following:
Changed the form helper /project/vendor/laravel/framework/src/llluminate/Html/FormBuilder.php in line 106
this:
$attributes['accept-charset'] = 'UTF-8';
to:
$attributes['accept-charset'] = 'latin1';
This way all of the forms were imputing to the database right symbols.
After this as my app is not big I have created helper to convert a given string from lating1 to UTF8 like this:
public static function convertToUtf8($string){
return $converted = iconv ('latin1' , 'utf-8' , $string);
}
And it's all working fine :)
I used MWDumper - http://www.mediawiki.org/wiki/Mwdumper - to import the xml dump of the English Language Wiktionary (specifically the file named enwiktionary-20120930-pages-meta-current.xml,) to my local server.
I have found that under the Translations section (on each page for each English word,) next to the name of each language where I should be able to see the definition in a foreign language, I instead see Template:Tø, Template:T+, or Template:T- and I am not sure why this is.
As an experiment, I also used WikiTaxi - http://www.yunqa.de/delphi/doku.php/products/wikitaxi/index - with the exact same XML dump and did not have this problem when viewing under WikiTaxi.exe.
I have been searching through mediawiki.org looking for the answer, but have so far not been successful.
Okay, I found out that MWDumper did the right thing importing the xml dump. All the translations are there. I just had to click on the Template:T+, Template:T- and Template:Tø links and add a template according to the instructions at http://www.mediawiki.org/wiki/Templates.
I'm thoroughly confused about how to read/write into igraph's Python module. What I'm trying right now is:
g = igraph.read("football.gml")
g.write_svg("football.svg", g.layout_circle() )
I have a football.gml file, and this code runs and writes a file called football.svg. But when I try to open it using InkScape, I get an error message saying the file cannot be loaded. Is this the correct way to write the code? What could be going wrong?
The write_svg function is sort of deprecated; it was meant only as a quick hack to allow SVG exports from igraph even if you don't have the Cairo module for Python. It has not been maintained for a while so it could be the case that you hit a bug.
If you have the Cairo module for Python (on most Linux systems, you can simply install it from an appropriate package), you can simply do this:
igraph.plot(g, "football.svg", layout="circle")
This would use Cairo's SVG renderer, which is likely to generate the correct result. If you cannot install the Cairo module for Python for some reason, please file a bug report on https://bugs.launchpad.net/igraph so we can look into this.
(Even better, please file a bug report even if you managed to make it work using igraph.plot).
Couple years late, but maybe this will be helpful to somebody.
The write_svg function seems not to escape ampersands correctly. Texas A&M has an ampersand in its label -- InkScape is probably confused because it sees & rather than &. Just open football.svg in a text editor to fix that, and you should be golden!