How to extract data from a PDF? - sql-server-2008

Question

My company receives data from an external company via Excel. We export this into SQL Server to run reports on the data. They are now changing to PDF format, is there a way to reliably port the data from the PDF and insert it into our SQL Server 2008 database?
Would this require writing an app or is there an automated way of doing this?

Answer 1

As already mentioned - you will have to write an app to do this, but ideally you would be able to get the raw data from the external company rather than having to process the PDF.
However, if you do want to extract the data from the PDF, I've used iText and found it to be very powerful, reliable and most importantly - free. It comes in Java and .Net flavours - iTextSharp is the .Net version. It allows you to programatically manipulate PDF documents and it will expose the contents of the PDF to the application that you write.

Answer 2

It all depends on how they've included the data within the PDF. Generally speaking, there's two possible scenarios here:
The data is just a text object within a PDF. You'll need to use a tool to extract the text from the PDF then insert it into your database.
The data is contained within form fields in a PDF. You'll need to use a tool to extract data from the form fields and insert it into your database.
Hopefully scenario #2 applies to you because this is precisely what PDF forms are designed for. Scenario #1 is really just a hack that you'd only use if you didn't have any other options. Extracting plain text from a PDF isn't as easy or accurate as you might expect.
If you're receiving a PDF form then all you need to do is match up the right fields in the PDF form with the corresponding fields in your database and then suck in the data. This process could be entirely automated if you wrote your own application.
Would this require writing an app or
is there an automated way of doing
this?
Yes, both of these options would require writing an app or buying an app. If you write your own app then you'll need to find a third-party PDF library that supports retrieving data from form fields or extracting text from a PDF.

Answer 3

Disclaimer: I am affiliated with the makers of ByteScout PDF Extractor SDK tool
Just wanted to share some additional real-life scenarios for text data extraction from PDF:
Scanned image with no searchable text: should be processed by OCR engine (like free Tesseract from Google)
XFA forms: it is the subset of PDF which is supported mostly by Adobe tools. But the data can be extracted as XML data with low level PDF processing tools like iTextSharp or similar tools.
ZUGFeRD PDF files which are just PDF documents with the copy of a form data attached as XML file (which can be extracted with tools like this)
Text incorrectly encoded by some PDF generators (can be restored via OCR engine with some acceptable error rate though).

Answer 4

Using ItextSharp, do the following
using System;
using System.Configuration;
using System.Data.SqlClient;
using System.IO;
using System.Text;
using iTextSharp.text.pdf;
protected void BtnSubmit_Click(object sender, EventArgs e)
{
String FilePath = #"GetFilePath";
StringBuilder sb = new StringBuilder();
PdfReader reader = new PdfReader(FilePath);
PdfStamper myStamp = new PdfStamper(reader, new FileStream(FilePath + "_TMP", FileMode.Create));
AcroFields form = myStamp.AcroFields;
if (form.GetField("GetFieldIdFromPDF") != null)
sb.Append(form.GetField("GetFieldIdFromPDF").ToString());
}

Answer 5

I think you will have to write an application for this. This question talks about extracting data from PDF. After this you can export the data to excel format so that you can preserve the existing import format.

Answer 6

Look for information on "Scraping" the data from the PDF. I believe Adobe has some tools that allow you to do this for simple text but I've not used them.
Honestly though, I would try to do anything you can to get this data in a raw format from your vendor.