Spark can't get delimiter for CSV file - csv

Question

I have a CSV file like this CSV read by pandas like this
But when I read it with PySpark, it turned out like this
CSV read by PySpark
What's wrong with the delimiter in Spark and how can I fix it?

Answer 1

From the posted images, %2C, which is URL encode equivalent of ,, seems to be your delimiter.
Set delimiter to %2C and also use header option:
df = spark.read.option("header",True).option("delimiter", "%2C").csv(path)
Input CSV File:
date%2Copening%2Chigh%2Clow%2Cclose%2Cadjclose%2Cvolume
2022-12-09%2C100%2C101%2C99%2C99.5%2C99.5%2C10000000
2022-12-09%2C200%2C202%2C199%2C199%2C199.1%2C20000000
2022-12-09%2C300%2C303%2C299%2C299%2C299.2%2C30000000
Output dataframe:
+----------+-------+----+---+-----+--------+--------+
|date |opening|high|low|close|adjclose|volume |
+----------+-------+----+---+-----+--------+--------+
|2022-12-09|100 |101 |99 |99.5 |99.5 |10000000|
|2022-12-09|200 |202 |199|199 |199.1 |20000000|
|2022-12-09|300 |303 |299|299 |299.2 |30000000|
+----------+-------+----+---+-----+--------+--------+