I copied a file from my ipad using iFunBox and I tried opened it with:
Microsoft Excel
FreeFileViewer
Notepad
Notepad++
All of them shown nothing File (.csv) Everytime open it using those 4 programmes this is what I get:
] 1 ˆ&ÓMF.¶•J¡ ñßR)åÙ¥e
ò¦Øœsl\V{`¸u°N±vI¦½™êk4“‡-m Ùò¾ˆÚär'²ø¢å:š nàÊ#‹8yH#uä³éc‹übuÙÀÙHIˆ[g¿^¶$9¹asï³`‰;*ý:Ôê]ÓlÆFJŠzÆy l[
†nÒdzÙ› ™uê^ï:—ŽœÏ·`ð€‡°XR…Hù™6è³ß×+¥Û/‹ÐÉÞ‡çååw¿öùr~uóçˆKÝcýœó©û$R0áÓ)`´Á°ŠOŸ¸hÉw”ÕÌ$¾S|åÚ†G¾Ùú¿#/¯J(Ádž‡Úøo䇙ñyŠÃ©A°6ÀÊ¿©þFDBº·SáóDEa±€o¹Nê¦ä~iÿñÍQ‹òÊun¢ç,Æâðe IéŒ
The file seems to be binary and not CSV. CSV is a plain text file which depending on the text encoding may show some strange characters, but mainly plainly readable. The file above is a kind of OMF file, whatever that is.
Related
I have a SSIS data flow task that reads from a CSV file and stores the results in a table.
I am simply loading the CSV file by rows (not even seperating the columns) and dumpting the entire row to the database, very simple process.
The file contains UTF-8 characters, and the file also has the UTF BOM already as I verified this.
Now when I load the file using a flat file connection, I have the following settings currently:
Unicode checked
Advanced editor shows the column as "Unicode text stream DT_NTEXT".
When I run the package, I get this error:
[Flat File Source [16]] Error: The data type for "Flat File
Source.Outputs[Flat File Source Output].Columns[DataRow]" is DT_NTEXT,
which is not supported with ANSI files. Use DT_TEXT instead and
convert the data to DT_NTEXT using the data conversion component.
[Flat File Source [16]] Error: Unable to retrieve column information
from the flat file connection manager.
It is telling me to use DT_TEXT but my file is UTF-8 and it will loose its encoding right? Makes no sense to me.
I have also tried with the Unicode checkbox unchecked, and setting the codepage to "65001 UTF-8" but I still get an error like the above.
Why does it say my file is an ANSI file?
I have opened my file in sublime text and saved it as UTF-8 with BOM. My preview of the flat file does show other languages correctly like Chinese and English combined.
When I didn't check Unicode, I would also get this error saying the flat files error output column is DT_TEXT and when I try and change it to Unicode text stream it gives me a popup error and doesn't allow me to do this.
I have faced this same issue for years, and to me it seems like it could be a bug with the Flat File Connection provider in SQL Server Integration Services (SSIS). I don't have a direct answer to your question, but I do have a workaround. Before I load data, I convert all UTF-8 encoded text files to UTF-16LE (Little Endian). It's a hassle, and the files take up about twice the amount of space uncompressed, but when it comes to loading Unicode into MS-SQL, UTF-16LE just works!
With regards to the actual conversion step I would say that is for you to decide what will work best in your workflow. When I have just a few files then I convert them one-by-one in a text editor, but when I have a lot of files then I use PowerShell. For example,
Powershell -c "Get-Content -Encoding UTF8 'C:\Source.csv' | Set-Content -Encoding Unicode 'C:\UTF16\Source.csv'"
My apologies if a solution has been provided elsewhere. I have searched and could not find anything similar to what I am experiencing. I am trying to upload categories on a Magento CE 2.1.1 website. I have a file with almost 4000 categories and sub categories and the only practical way is to upload via a csv file.
I downloaded a sample file to use and when I upload the same sample file it's working fine when I click Check data" button. However, when I replace the values on the rows with my own and save the file as csv with UTF-8 text encoding, I am getting an error message below. This is also happening when I save the file as csv even without changing the values. I have tested this with a csv file saved from both Mac Numbers and Windows Excel.
I only need to upload Categories (and not products) but I am not sure if this is possible.
File links:
Importing
Not importing
Actual project sample file
The files are quite similar but strangely one is working and the other is not.
Error
We can't find required columns: sku.
Column names: "sku;store_view_code;attribute_set_code;product_type;categories;product_websites;name;description;short_description;weight;product_online;tax_class_name;visibility;price;special_price;special_price_from_date;special_price_to_date;url_key;meta_title;meta_keywords;meta_description;base_image;base_image_label;small_image;small_image_label;thumbnail_image;thumbnail_image_label;swatch_image;swatch_image_label;created_at;updated_at;new_from_date;new_to_date;display_product_options_in;map_price;msrp_price;map_enabled;gift_message_available;custom_design;custom_design_from;custom_design_to;custom_layout_update;page_layout;product_options_container;msrp_display_actual_price_type;country_of_manufacture;additional_attributes;qty;out_of_stock_qty;use_config_min_qty;is_qty_decimal;allow_backorders;use_config_backorders;min_cart_qty;use_config_min_sale_qty;max_cart_qty;use_config_max_sale_qty;is_in_stock;notify_on_stock_below;use_config_notify_stock_qty;manage_stock;use_config_manage_stock;use_config_qty_increments;qty_increments;use_config_enable_qty_inc;enable_qty_increments;is_decimal_divided;website_id;related_skus;related_position;crosssell_skus;crosssell_position;upsell_skus;upsell_position;additional_images;additional_image_labels;hide_from_product_page;bundle_price_type;bundle_sku_type;bundle_price_view;bundle_weight_type;bundle_values;bundle_shipment_type;associated_skus" are invalid
This might be because you opened the file in Excel which will add a BOM to the start of the file. When the Magento importer tries to read the file, it expects the first header/cell to say sku, but it instead sees the BOM.
Two ways to solve this:
1) Don't open it in excel - use Google sheets, or a text editor if you are feeling brave,
2) If you opened the file in excel, close it, open it in notepad++, click encoding up the top and set to "Encode in UTF-8" (NOT "Encode in UTF-8-BOM"). Then save and you are good to go.
I'm having issues with a Open Office. I'm trying to export some files as CSV text and it appears that underscores and perhaps some other characters are being replaced. For instance, I created this file in a text editor and saved it as test.csv:
"this_value","is_replaced"
I then opened the file into Open Office as a spreadsheet. It looked fine. Then I saved it as a new CSV file named test-export.csv. When I open the new file in a text editor, the underscores are replaced with "+AF8-", and the comma is replaced with a tab.
this+AF8-value is+AF8-replaced
This change makes the file useless for my purposes. Any idea what is causing this?
Apparently the problem is caused when you import the CSV file into Open Office. When you import it you have a choice which character set to use. I used UTF-7 when I should have used UTF-8. When I import using UTF-8, it then saves the file as I expect it should.
I'm having an issue where I can't get this symbol "℗" to render in a .tsv file. I'm using powershell to add data to my .tsv that has copyright info, so I need to have it copy over correctly. I use add-content -path C:\blah and I include the -encoding parameter at the end, but all of the encoding choices I've tried cannot render this sound recording copyright symbol. Anyone have any idea if this can work? UTF8 and UTF32 render the © symbol correctly, for what it's worth. What's also sort of funny is that Powershell ISE can render the symbol correctly.
Thanks for any help.
Edit: I'm thinking now it may just be a limitation of the .tsv file? I just tried opening the .tsv in excel, pasting the "℗" symbol into an empty cell, and when I save and re-open the file, the "℗" is replaced by a "?".
Edit 2: If I use import-csv -path -delimiter and import the .tsv, the symbol does render correctly in Powershell. I would like it to render correctly in Excel if that's possible. I also tried to load it using Google Sheets but it had the same problem.
This turns out to be sort of a workaround I think, but the end result is what I wanted.
There seems to be some type of problem with excel and the encoding of .tsv, .csv, etc. So what I found worked for me was opening whatever file I wanted (in Excel), saving as a "Unicode .txt", then just renaming the file extension from .txt to .tsv. The file still opens correctly as a .tsv spreadsheet, but for whatever reason the encoding works now so that the "℗" saves correctly. And it looks like I only need to do this once and I can keep appending to the same .tsv. Strange solution, but I'll take it.
And just to clarify, if I open the file as a .txt in Excel, it also retains the correct formatting. I just changed the extension to .tsv so that a double click will open to Excel rather than notepad. I believe the file is technically still saved as a Unicode .txt file.
I am having difficulty mapping a CSV file with the Coded UI test method. This is most likely a stupid question but I cannot seem to find a solution for my problem, at least not one that works. I have made sure to set the property of the CSV file to Copy always.
I have also imported the CSV file by writing the following line above the test method.
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", "|DataDirectory|\\Data\\login.csv", "login#csv", DataAccessMethod.Sequential), DeploymentItem("login.csv"), TestMethod]
The file name is login.csv and it resides in the Data folder.
The test will compile without any problem but once the test executes the fields that should receive input from the CSV file are left empty and the execution is interrupted. I've tried replacing the data from the CSV file by using Strings and it works perfectly fine. The piece of code I am using to import each parameter is:
TestContext.DataRow["Username"].ToString()
Also, the CSV file contains something along the following lines:
Username,Password,Fullname
admin#mail.com,password,Admin
Is there anyone who can point what it is I am forgetting.
Update: I pinpointed the issue, it seems like the issue only revolves around the first column in the csv file. When I try to import any of the other values it works perfectly fine.
Some text files start with a Byte Order Mark (BOM). The CSV reader within Coded UI does not handle the BOM and treats it as part of the first field name. The screen shot below shows the debug trace of a CSV file with a BOM and that same file shown in Notepad++. The DataRow.ItemArray[...] values are as expected. The DataRow.Table.Columns.ResultsView[...] shows the field names, but the first field name includes the BOM.
This CSV file with a BOM was created in Visual Studio using Solution Explorer => Add => New item => C# => General => Text file. Previously I have created a spread sheet with Microsoft Excel and saved it as a CSV file, that file did not have a BOM. I have also created files with Notepad++ and saved as CSV and they did not have a BOM. It appears that Visual Studio creates files with a BOM but when editing CSV files it does not add a BOM.
Visual Studio can create files with the correct encoding. Within "Step 2 - Create a data set" of this Microsoft page it states the text below. (Thanks also to Holistic Developer for providing very similar details in a comment.):
It is important to save the .csv file using the correct encoding. On the FILE menu, choose Advanced Save Options and choose Unicode
(UTF-8 without signature) – Codepage 65001 as the encoding.
For Visaul Studio 2010, i could solve issue be selecting "Western European (Windows) - Codepage 1252" encoding for CSV files.
Summary of steps:
In visual studio 2010, Open CSV file > Go to File menu > Select " Advanced Save Options" > Select "Western European (Windows) - Codepage 1252" > Save.
This should help.
This is not the best solution but its kind of a workaround. I simply set the first element to something random and since I don't need access to the first element it doesn't matter that I don't have access to it.
If anyone finds a correct way to solve this problem I'd be grateful for your solution.