Merging two csv files, can't get rid of newline - csv

Question

I am merging two csv files. For simplicity, I am showing relevant columns only. There are more than four columns in both files.
file_a.csv
col2, col6, col7, col17
a, b, c, 145
e, f, g, 101
x, y, z, 243
file_b.csv
col2, col6, col7, col17
a, b, c, 88
e, f, g, 96
x, k, l, 222
Output should look like this:
col2, col6, col7, col17, col18
a, b, c, 145, 88
e, f, g, 101, 96
So col17 of file_b is added to file_a as col18 when the contents of col2, col6 and col7 match.
I tried this:
awk -F, 'NR == FNR {a[$2,$6,$7] = $17;next;} {if (! (b = a[$2,$6,$7])) b = "N/A";print $0,FS,b;}' file_a.csv file_b.csv > out.csv
The output looks like this:
col2, col6, col7, col17,
, col18
a, b, c, 145
, 88
e, f, g, 101
, 96
So the column 17 from file_b I am trying to add does get added but shows up on a new line.
I think this is because there are carriage returns after each line of file_a and file_b. In Notepad++, I can see CRLF. But I can't get rid of them. Also, I would rather not go through two steps: getting rid of carriage returns first and then merging. Instead, if I can bypass the carriage returns during the merge, it will be much faster.
Also, I will appreciate it if you could tell me how to get rid of the spaces before and after the comma separating the merged column. Note that I put spaces between the columns and commas for the other columns for better readability. That is not how it is in the actual files. But there are indeed spaces between col17 and "," and col18 in the merged file and I don't know why.
If you insist on marking this as a duplicate, kindly explain in a comment below how the answers to the previous question(s) address my issue. I tried figuring it out from those previous similar questions and I failed.

Answer 1

Try this please (GNU awk):
awk -F, -v RS="[\r\n]+" 'NR == FNR {a[$2,$6,$7] = $17;next;} {b=a[$2,$6,$7]; print $0 FS (b? b : "N/A")}' file_a.csv file_b.csv
The thing you have problem at:
1. Carriage returns, by RS="[\r\n]+", it will treat multiple newlines, including \r and \n as line separators. Note this will also ignore empty lines, if you don't want to, change to RS="\r\n".
2. The spaces, that's because awk's default OFS is a space. And when you print, you used ,, this will add spaces between them. Just use space or sometime just write them together will do, they will be concatenated.

Answer 2

Could you please try following.
awk -v RS="[\r\n]+" '
BEGIN{
SUBSEP=OFS=", "
}
FNR==NR{
if(FNR==1){
header=$0
}
a[$1,$2,$3]=$4
next
}
FNR==1 && FNR!=NR{
split(header,array,", ")
sub(/[a-zA-Z]+/,"",array[4])
print header,"col"array[4]+1
next
}
a[$1,$2,$3]{
print $0,a[$1,$2,$3]
}' b.csv a.csv
What above code does:
1- Seems like you may have carriage returns in your Input_file(s) so I have made \r\n as record separator(in case you want to remove carriage returns then try tr -d '\r < a.csv > temp && mv temp a.csv and do for other fiels too).
2- This will create header also as per your file's last column too.

Answer 3

with Miller (http://johnkerl.org/miller/doc)
mlr --csv join -j col2,col6,col7 --lp l --rp r -f file_a.csv \
then unsparsify --fill-with "" \
then rename lcol17,col17,rcol17,col18 file_b.csv
you have
col2,col6,col7,col17,col18
a,b,c,145,88
e,f,g,101,96
I have used as input
# file_a.csv
col2,col6,col7,col17
a,b,c,145
e,f,g,101
x,y,z,243
# file_b.csv
col2,col6,col7,col17
a,b,c,88
e,f,g,96
x,k,l,222

Answer 4

Since you wanted to get of spaces between the delimiter ,, you can try this Perl solution, that removes the spaces while splitting.
The answer assumes you have \r in the files. I have used -vT option for cat to show that the carriage return exists
$ cat -vT file_a.csv
col2, col6, col7, col17^M
a, b, c, 145^M
e, f, g, 101^M
x, y, z, 243^M
$ cat -vT file_b.csv
col2, col6, col7, col17^M
a, b, c, 88^M
e, f, g, 96^M
x, k, l, 222^M
$
$ perl -lne 'BEGIN { %kv=map{chomp;chop;#a=split(/\s*,\s*/);"$a[0],$a[1],$a[2]"=>"$a[3]"} qx(cat file_b.csv) } chop;#b=split(/\s*,\s*/);$x="$b[0],$b[1],$b[2]"; print "$x,$b[-1],",$kv{$x} if $kv{$x} ' file_a.csv
col2,col6,col7,col17,col17
a,b,c,145,88
e,f,g,101,96
$