Equalize number of columns in csv data [closed] - csv

Question

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I'm parsing some contact data from a PDF-file converted to csv which has resulted in a different column count per row based on missing entries.
Is there a way to correct this using sed, awk, cut etc by making sure some columns that are easy to pattern match up - fx making sure the email-addresses are in the same columns when available and others like "Lifetime member" or "Guest" when email is not available?
The first column is a person/company's name but the rest is arbitrary. The point is to extract contact information(like email, phone number etc) and put it in the same columns, when available.
My idea would be to check if the email is in the 6th column and if not then add a number of empty columns before it etc
Example data:
Steve Smith;9828;1;[phone];Guest;steve#example.org;1;1 12th st;48572 Nowhere
Steve Jobs;[phone];noreply#example.org;1;48572 Nowhere
John Smith;9828;1;[phone];Lifetime member;1;1 23rd st;48572 Nowhere
Peter Blavounius;2312;peter#blavounius.com
Wanted output:
Steve Smith;9828;1;[phone];Guest;steve#example.org;1;1 12th st;48572 Nowhere
Steve Jobs;[phone];;;;noreply#example.org;1;;48572 Nowhere
John Smith;9828;1;[phone];Lifetime member;1;1 23rd st;48572 Nowhere
Peter Blavounius;2312;;;;peter#blavounius.com

Answer 1

This will get you started but it is not complete, you still need to identify other fields, all I've done so far is identify a couple of the fields to show you the approach:
$ cat tst.awk
BEGIN {
FS=OFS=";"
ARGV[ARGC] = ARGV[ARGC-1]
ARGC++
}
{
name = tel = email = digs4 = ""
for (i=1;i<=NF;i++) {
if (i == 1) {
name=$i; $i=""; nameFld = 1
}
else if ($i ~ /^\+/) {
tel=$i; $i=""; telFld = (i > telFld ? i : telFld)
}
else if ($i ~ /#/) {
email=$i; $i=""; emailFld = (i > emailFld ? i : emailFld)
}
else if ($i ~ /^[0-9]{4}$/) {
digs4=$i; $i=""; digs4Fld = (i > digs4Fld ? i : digs4Fld)
}
}
maxFlds = (NF > maxFlds ? NF : maxFlds)
}
NR>FNR {
for (i=1;i<=maxFlds;i++) {
if (i == nameFld) { $i = name }
else if (i == telFld) { $i = tel }
else if (i == emailFld) { $i = email }
else if (i == digs4Fld) { $i = digs4 }
else { $i = $i } # make sure null fields are present
}
print
}
.
$ awk -f tst.awk file
Steve Smith;9828;1;[phone];Guest;steve#example.org;1;1 12th st;48572 Nowhere
Steve Jobs;;;[phone];48572 Nowhere;noreply#example.org;;;
John Smith;9828;1;[phone];Lifetime member;;1 23rd st;48572 Nowhere;
Peter Blavounius;2312;;;;peter#blavounius.com;;;
It does 2 passes on your input file - the first to identify that largest field number that matches each regexp as that's where you want every field matching that regexp to appear in the output, and the second to identify the fields, clear out their location in the record, and then place every field in the right location.
You could identify what a field means by matching it's context to a regexp like above or by its fixed position in the line (e.g. the persons name is always in field 1) or by it's relative position to something else (e.g. a single digit occurring before vs after the email address or before/after the 3rd field number or....)
Hope it makes sense. Add some printfs and play with it a bit and ask questions if you're confused after that.