I created a table by using Postgresql after that I tried to copy the json document inside of that table with:
\copy document_json FROM '/path/to/document.json';
However it gives an error such as: ERROR: character with byte sequence 0x9e in encoding "WIN1254" has no equivalent in encoding "UTF8"
CONTEXT: COPY document_json, line 1
I couldn't find any explanation for this error. I would appreciate any help. Thank you!
Related
I am trying to import csv file to table in postgres using COPY command. I have problem that one column is of json data type. I tried to escape json data in csv using dollars ($$...$$) docu_4.1.2.2.
This is first line of csv:
3f382d8c-bd27-4092-bd9c-8b50e24df7ec;370038757|PRIMARY_RESIDENTIAL;$${"CustomerData": "{}", "PersonModule": "{}"}$$
This is command used for import:
psql -c "COPY table(id, name, details) FROM '/path/table.csv' DELIMITER ';' ENCODING 'UTF-8' CSV;"
This is error I get:
ERROR: invalid input syntax for type json
DETAIL: Token "$" is invalid.
CONTEXT: JSON data, line 1: $...
COPY table, line 1, column details: "$${CustomerData: {}, PersonModule: {}}$$"
How should I escape/import json value using COPY? Should I give up and use something like pg_loader instead? Thank you
In case of failing with importing the JSON data please give a try to the following setup - this worked for me even for quite complicated data:
COPY "your_schema_name.yor_table_name" (your, column_names, here)
FROM STDIN
WITH CSV DELIMITER E'\t' QUOTE '\b' ESCAPE '\';
--here rows data
\.
i am getting the following error if i run a copy command to copy contents of a .csv file in s3 to a table in redshift.
error:"String length exceeds DDL length".
i am using following copy command:
COPY enjoy from 's3://nmk-redshift-bucket/my_workbook.csv' CREDENTIALS 'aws_access_key_id=”****”;aws_secret_access_key=’**** ' CSV QUOTE '"' DELIMITER ',' NULL AS '\0'
i figured lets open the link given by s3 for my file through was console.
link for the work book is :
link to my s3bucket cvs file
the above file is filled with many weird characters i really don't understand.
the copy command is taking these characters instead of the information i have entered in my csv file.So hence leading to string length exceeded error.
i use sql workbench to query.My 'stl_load_errors' table in redshift has raw_field_values component similar to the chars in the link i mentioned above, thats how i got to know how its taking in the input
i am new to aws and utf-8 configs. so please i appreciate help on this
The link you provide points to a .xlsx file (but has a .csv extension instead of .xlsx), which is actually a zip file.
That is why you see those strange characters, the first 2 being 'PK', which means it is a zip file.
So you will have to export to .csv first, before using the file.
I'm reading some values from a csv like this:
But I'm getting some strange extra characters at the beginning of the ID value like this:
The CSV file is just an id, a name and line breaks. Why do I get these other characters?
The specific characters you see ( or EF BB BF in hex) are Byte order mark for UTF-8. So it is coming from your CSV file, likely somehow added on saving the file. So try to set File encoding parameter to UTF-8, it should help.
In my case I have selected UTF-8 as encoding, It worked fine for few parameters, but I still observed  before one parameter i.e URL.
CSV used: url,id,token
Url looks like: abc.def.ghi.jkl.com
Id looks like: jhas880ad
token is JWT token
I observed following exception in jMeter response:
java.net.UnknownHostException: ?abc.def.ghi.jkl.com
at java.net.InetAddress.getAllByName0(Unknown Source)
at java.net.InetAddress.getAllByName(Unknown Source)
at java.net.InetAddress.getAllByName(Unknown Source)
at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
at
Then after a long research, following solution worked for me:
Jmeter put  as prefix for the value of 1st variable taken from CSV file
Thus I used one dummy entry in beginning of csv file and wrote dummy variable name in jMeter csv data set config
click to view image
Updated CSV looked like: dummy,url,id,token
This solution might not be practical for huge data but if you have very few records then you can consider this.
Also, if someone is aware of another workaround, feel free to post.
If you are exporting a ResultSet from DBeaver to a CSV, once you get to the "Output" tab, make sure to select "UTF-8" in the "Encoding" list AND to uncheck the "Insert BOM" box. That worked for me.
I am trying to load large volume of data into graph using CSV Load script (xyx.cpl) and Neo4jShell.
Mostly it is doing well. Sometimes I am receiving following errors
Cannot merge node using null property value ...
Error related to escape characters
So, seeking assistance to understand the best way to handle this issues in import script.
Thanks in advance
Cannot merge node using null property value
You can use a WITH statement to filter out rows that have a null value for the property you are using in the MERGE. For example:
LOAD CSV WITH HEADERS FROM "file:///file.csv" AS row
WITH row WHERE row.name IS NOT NULL
MERGE (p:Person {name: row.name})
SET p.age = row.age
...
Error related to escape characters
Can you be a bit more specific about the error you are getting / show a Cypher and data example?
Without seeing your specific error / code here is some info that might help:
the character for string quotation within your CSV file is a double quote "
the escape character is \
more info and some examples here and here
I am currently working on a project that requires a large data migration for a company. We are in the process of planning and testing data imports from an existing Access database to a MySQL database for a CRM they will be using.
We have encountered errors with importing (using Load Data Infile) exported data in .csv format, when the records have accented or special characters due to the files being imported being in ANSI format (the rest of the MySQL database is all in UTF8). I managed to fix this issue by using the Convert to UTF8 functionality in Notepad++, but this was before I knew we needed the existing primary key ID's from the Access database to be imported as well.
Doing this same process with the added ID's causes a MySQL error to throw:
Error Code: 1366. Incorrect integer value: '135' for column 'id' at row 1
Is there a way to convert all this data to UTF8 without having integer values throw errors?
Convert the file to UTF-8 without BOM and try again :)
The trick is that at beginning of the UTF-8 file there is a BOM sequence and your number 135 at the beginning of the file is actually 0xEF 0xBB 0xBF 1 3 5 what causes error in TSV importer unaware of UTF-8.