Reading CSV with list valued columns in rust - csv

Question

I'm trying to use csv and serde to read a mixed-delimiter csv-type file in rust, but I'm having a hard time seeing how to use these libraries to accomplish it. Each line looks roughly like:
value1|value2|subvalue1,subvalue2,subvalue3|value4
and would de-serialize to a struct that looks like:
struct Line {
value1:u64,
value2:u64,
value3:Vec<u64>,
value4:u64,
}
Any guidance on how to tell the library that there are two different delimiters and that one of the columns has this nested structure?

Answer 1

Ok, I'm still a beginner in Rust so I can't guarantee that this is good at all- I suspect it could be done more efficiently, but I do have a solution that works-
use csv::{ReaderBuilder};
use serde::{Deserialize, Deserializer};
use serde::de::Error;
use std::error::Error as StdError;
#[derive(Debug, Deserialize)]
pub struct ListType {
values: Vec<u8>,
}
fn deserialize_list<'de, D>(deserializer: D) -> Result<ListType , D::Error>
where D: Deserializer<'de> {
let buf: &str = Deserialize::deserialize(deserializer)?;
let mut rdr = ReaderBuilder::new()
.delimiter(b',')
.has_headers(false)
.from_reader(buf.as_bytes());
let mut iter = rdr.deserialize();
if let Some(result) = iter.next() {
let record: ListType = result.map_err(D::Error::custom)?;
return Ok(record)
} else {
return Err("error").map_err(D::Error::custom)
}
}
struct Line {
value1:u64,
value2:u64,
#[serde(deserialize_with = "deserialize_list")]
value3:ListType,
value4:u64,
}
fn read_line(line: &str) -> Result<Line, Box<dyn StdError>> {
let mut rdr = ReaderBuilder::new()
.delimiter(b'|')
.from_reader(line.as_bytes());
let mut iter = rdr.deserialize();
if let Some(result) = iter.next() {
let record: Line = result?;
return Ok(Line)
} else {
return Err(From::from("error"));
}
}
[EDIT]
I found that the above solution was intolerably slow, but I was able to make performance acceptable by simply manually deserializing the nested type into a fixed size array by-
#[derive(Debug, Deserialize)]
pub struct ListType {
values: [Option<u8>; 8],
}
fn deserialize_farray<'de, D>(deserializer: D) -> Result<ListType, D::Error>
where
D: Deserializer<'de>,
{
let buf: &str = Deserialize::deserialize(deserializer)?;
let mut split = buf.split(",");
let mut dest: CondList = CondList {
values: [None; 8],
};
let mut ind: usize = 0;
for tok in split {
if tok == "" {
break;
}
match tok.parse::<u8>() {
Ok(val) => {
dest.values[ind] = Some(val);
}
Err(e) => {
return Err(e).map_err(D::Error::custom);
}
}
ind += 1;
}
return Ok(dest);
}