SparkSession CSV load sometimes doesn't work - csv

Question

I have 3 CSV files which are delimited by ;
I am able to read 2 of these files with SparkSession CSV loader, but the third file causes an ArrayIndexOutOfBoundsException. When I inspect the three files manually, it looks like they all adhere to the exact same format, so I'm having trouble figuring out what causes SparkSession's CSV loader to fail. It also looks like Spark is not offering me any ways to debug its CSV loader.
Here is the code which works for 2 out of 3 files:
spark
.read
.format("csv")
.option("delimiter", ";")
.load("data/bx/BX-Users.csv")
Part of the stack trace:
Caused by: java.lang.ArrayIndexOutOfBoundsException: 62
at org.apache.spark.unsafe.types.UTF8String.numBytesForFirstByte(UTF8String.java:191)
at org.apache.spark.unsafe.types.UTF8String.numChars(UTF8String.java:206)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$10$$anon$1.hasNext(WholeStageCodegenExec.scala:614)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:253)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:247)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:830)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:830)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:109)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Part of BX-Users.csv:
"1";"nyc, new york, usa";"NULL"
"2";"stockton, california, usa";"18"
"3";"moscow, yukon territory, russia";"NULL"
"4";"porto, v.n.gaia, portugal";"17"