Handle CSV data: Separate data with a comma - csv

Question

I need to handle the data from a CSV list. I need to import this list, but for that, I need to join the data, separated by commas.
Currently the list looks something like this:
COMPANY;CITY;COUNTRY
Comp1;curitiba;brazil
Comp2;curitiba;brazil
Comp3;detroit;usa
Comp4;detroit;usa
What I need is to group the values, separator by comma, according to the city, exactly as follows:
COMPANY;CITY;COUNTRY
Comp1,Comp2;curitiba,curitiba;brazil,brazil
Comp3,Comp4;detroit,detroit;usa,usa
The list has thousands of lines, which makes it impossible to do it manually.
Would anyone know a way to do this with some tool?

Answer 1

You could use some SQL aggregation function.
In sqlite in example, starting from this kind of table, named "input"
+---+---+---+
| a | b | c |
+---+---+---+
| 2 | 3 | 3 |
| 1 | 3 | 4 |
| 2 | 5 | 7 |
+---+---+---+
and running
SELECT
a,GROUP_CONCAT(b) b,GROUP_CONCAT(c) c
FROM
input
group by a
you will have
+---+-----+-----+
| a | b | c |
+---+-----+-----+
| 1 | 3 | 4 |
| 2 | 3,5 | 3,7 |
+---+-----+-----+
The GROUP_CONCAT function is in a lot of db systems (not only sqlite).

Answer 2

Using Miller (https://github.com/johnkerl/miller) is very easy.
Running
mlr --csv --fs ";" nest --implode --values --across-records --nested-fs "," -f COMPANY input.csv
you will have
COMPANY;CITY;COUNTRY
Comp1,Comp2;curitiba;brazil
Comp3,Comp4;detroit;usa
Some notes:
--fs ";", to set the field separator of the file;
nest --implode --values --across-records, to set the implode nesting across the records;
--nested-fs ",", to set the nested field separator;
-f COMPANY, to set the field to implode.

Answer 3

If I'm understanding you correctly and you just want every two rows to be combined into a single row, then the following will get you where you need to go. Otherwise, it should be enough to introduce you to Power Query and you can adjust from what this is to what you need.
From within a new workbook in Excel...
Do Data - New Query - From File - From CSV from the Microsoft Excel ribbon. This will open up the "Power Query Editor"
Confirm/adjust to make sure it's importing your starting format correctly.
Rename the query you create to "RawImport". (Look for the "Name" property on the right sidebar.)
Create a new query that starts with the current one by, from the ribbon (in the Power Query editor), clicking Home - Manage - Reference
Click View and ensure that "Formula Bar" in the ribbon is checked.
Change the name of this query to "RawImportEvenOdd"
Click Add Column - Index Column from the ribbon. This should create a new "Index" column that starts with 0 and increments by 1.
Click Add Column - Custom Column from the ribbon. In the dialog box, give it a name of "IsEvenRowNum", and a formula of Number.Mod([Index], 2) = 0
Click the little "ABC 123" icon in the left edge of your new "IsEvenRowNum" column's title. Change it to a type of "True/False". This isn't necessary, but trust me and start your habit of strongly typing your data today. It will save you from yourself one day.
Create a new query that will be only the even rows by clicking Home - Manage - Reference
Rename this third query to "RawImportEvenOnly"
Click on the little filter button on the right edge of the "IsEvenRowNum" and filter your query to only show rows where IsEvenRowNum is true
Click Add Column - Custom Column and in the dialog box, name it "NewRowNum" with a formula of [Index]/2
Click Home - Choose Columns and change it so only COMPANY, CITY, COUNTRY, and NewRowNum are selected.
Double-click on the "COMPANY" column title and add "_E" to the end (renaming it to "COMPANY_E")
Repeat the same for CITY and COUNTRY
Create a new query that will be only the odd rows by clicking Home - Manage - Duplicate (note: Duplicate, not Reference)
Change the query name to "RawImportOddOnly"
Click on the Filtered Rows step in the right side-bar and then edit the displayed formula in the formula bar to change true to false
Click on the next step, "Added Column", and then edit the formula from = Table.AddColumn(#"Filtered Rows", "NewRowNum", each [Index]/2) to = Table.AddColumn(#"Filtered Rows", "NewRowNum", each ([Index]-1)/2)
In the last step, "Renamed Columns", change it so that it appends "_O" instead of "_E" to COMPANY, CITY, and COUNTRY.
If the left side-bar hasn't already expanded for you, you can expand it to see the queries you've made so far. Click the "RawImportEvenOnly" query to go back to editing it.
Click Home - Manage - Reference to start making your final query.
Change the name to whatever you like.
Click Home - Merge Queries and, in the dialog box,
Click on the "NewRowNum" column title to highlight it.
Change the dropdown box for the second table to be "RawImportOddOnly"
Click on the "NewRowNum" column title on this second table to highlight it.
Click Okay.
Okay, so now the value of our new column for any given row is a subtable of the corresponding row from RawImportOddOnly, and we need to expand it. Instead of a filter button, the button the right edge of the new "RawImportOnly" column is an expand button. Click it. In the dialog box, make sure only COMPANY_O, CITY_O, and COUNTRY_O are selected, deselect "Use original column name as prefix" at the bottom of the dialog box, then click "Ok"
Click Add Column - Custom Column, name it "COMPANY" with a formula of [COMPANY_E]&","&[COMPANY_O]
Click Add Column - Custom Column, name it "CITY" with a formula of [CITY_E]&","&[CITY_O]
Click Add Column - Custom Column, name it "COUNTRY" with a formula of [COUNTRY_E]&","&[COUNTRY_O]
Click Home - Choose Columns and change the dialog box so only COMPANY, CITY, AND COUNTRY are selected.
In the Home ribbon, click, the tiny little down-arrow in the "Close & Load" button and choose "Close & Load To", and choose "Only Create Connection" and click Ok.
Now you're back at Excel and nothing's happened (except for your new queries appearing in a side-bar)! This is because we didn't load anything into Excel, we only made query definitions. We did this because we just created a whole lot of queries and we didn't want to store the results of each and every one; we only want the last one! In the side-bar, right click on the last query you created, click "Load to..." and load it to a new worksheet. This will do all of the work we've set up, from the beginning... which is great because if you realize you want to change any part of how this was made, go ahead, then refresh! Maybe I misunderstood you and you needed to sort it first? Maybe you've changed your mind and want to separate by slashes instead of commas? Change the queries and refresh the resulting table, and your new set of steps are done.
Not described is how to get this into a the semi-colon delimited file you're looking for, but I assume you can take it from there?