Does anyone know of a good tool that converts .pst to .csv files through command line?
Can you assume Outlook is installed on the computer? If so, I believe it can be background scripted using OLE or something similar. I've done file conversions through Excel using Ruby that way.
And here's a Perl example
A solution I just stumbled across is:
libpst
It obviously doesn't convert straight to CSV, but it converts into a more manageable format.
Importing into Outlook and then exporting as CSV is still probably the quickest solution, but libpst would certainly be useful if all you have is the PST file and no Outlook.
one time only? or programmatically?
if one time only, import into a mail program that handles mbox (e.g. Thunderbird), at which point you just have text files, manipulate as desired.
otherwise, no idea, best of luck.
You can always write a .Net application using CDO, MAPI, OOM or Redemption, that does what you need.
I've written a complete Outlook exporter tool for my company, which you can view at http://www.tzunami.com
Related
How to generate DBF files with delphi?
More information:
I need a portable Database to run in a pendrive
I don't know if DBF is the better soluttion.
I like MySql but is hard to find a portable version
I am working with XML at this time but I don't know how to make a Query.
TDBF seems to be one of the most used (and still maintained) packages:
http://tdbf.sourceforge.net/
If you are open to alternative formats, I would strongly recommend you check out Firebird embedded. I'm pretty sure I loaded my database and my application on a flash drive as an experiment about two years ago and it ran fine. I stuck with DBF files for a long time, but once I made the switch, I would never go back. If you want to give this a try, consider using DBExpress and the Interbase driver. Although it's not officially supported, I have not found any incompatibilities. For a visual management tool, get the personal edition of IBExpert.
Not dealt with that in years!
Depending on your version of Delphi and what you want to do, you can either use the BDE or find some specialized components that read/write directly to DBF native file format.
Wow, I'm amazed this url is still valid!
If you want to create a DBF with the BDE API, you can use the techniques here:
http://info.borland.com/devsupport/bde/bdeapiex/dbicreatetable.html
However, depending on the size of the dataset you're going to have, this sounds like a single-user app, which would work just fine with a "MyBase" file -- another name for the files used by TClientDataSet. You can save it either as XML or binary (CDS) format by simply setting the TClientDataSet.FileName property, which will be used to read and write the dataset. You can even create nested datasets with this.
If you want to have the most efficient single user mode, also turn off the ChangeLog on the TClientDataSet.
procedure TFormCDSDataBug.ButtonOpenClick(Sender: TObject);
begin
ClientDataSet1.FileName := ExtractFilePath(Application.ExeName) + 'MyData.cds';
ClientDataSet1.LogChanges := False;
ClientDataSet1.Open;
end;
While I haven't used it, there is a component jdDbf which appears to be recently updated that appears that it will work. Something to be aware of is that the Dbf format has many flavors, however most of the variances appear to be in the index files so if you need to use it with another system you might have to rebuild them.
For a embedded database that works well on a usb drive, I would look at SQLite. There is a Delphi implementation DISQLite3 which has a free and pro version that would work well and comes highly recommended. There are many commercial quality programs currently using it, including the likes of FeedDemon.
I need to convert a very large file from DBF format to CSV format. I have tried Microsoft Excel to do the job, but the problem is that I cannot see more than 65500 registers when I open and export the file.
Microsoft Access couldn't open the file, too.
I have found on google some shareware tools, searching for "DBF to CSV". Have you tried any of these with very large files?
Also, any solution that could export to mysql or postgresql database formats will be welcome.
Thanks in advance for your responses, best regards,
https://github.com/SocialExplorer/FastDBF
"Also included here is a small utility that reads DBF files and outputs CSV files! "
go to http://www.the-oasis.net/ftpmaster.php3?content=ftputils.htm
look for this file dbx130.zip
Bytes: 125,478 Date: 1993-03-22
dbMAX is an xBASE utility that will allow complete multi-user access
to any xBASE databases and indexes. The program uses a CUA-type menu
system with Brief(R)-style hot keys and can browse databases in up to
250 moveable, sizable windows. Almost every Clipper(R)/dBASE(R)
command is available, allowing dbMAX to replace the dBASE
Assist/Control Center or Computer Associates' DBU utility. dbMAX also
has a partially open architecture, allowing programmers to create
their own menus and operate on dbMAX internal data structures.
this utility has a dos ui but it allows you via the Copy function on the menu to export entire dbf tables in SDF or CSV format. I personally know that it can handle a file with 3.8 million rows so it should be able to handle your table.
Use OpenOffice - Its free and can handle a lot of rows. With that many rows, you might need to split the file and then convert the pieces and then reassemble.
OpenOffice 3.0 Calc maxes out at 65K rows. I tried importing a large DBF into OpenOffice 3.0 Base but it handed the job off to Calc :-(
Alternative: if you have Python 2.4 to 2.6, I can send you a copy of my soon-to-go-public DBF-reading module plus a DBF-to-CSV script. To get my e-mail address, search for "John Machin xlrd" [xlrd is my Excel XLS-reading package].
What would be the easiest way to take a Outlook pst file and export all of the emails into a MySQL database?
Yikes. Probably the easiest would be to open your PST in Outlook, and use
File->Import and Export, Export to a File, Comma Seperated Values (Windows)
This creates a CSV file, which you can then pull into MySQL via mysqlimport.
If you need more information besides just the contents of the messages, you will need to tap into the store directly through various exotic means.
Powershell could be good for this? Eg enum emails in a folder, create sql insert for each, append insert to batch sql script:
$olApp = New-Object -com Outlook.Application
$namespace = $olApp.GetNamespace("MAPI")
$folder = $namespace.GetDefaultFolder(1)
$folder.Items | %{
"insert into MyTable (MyCol1, MyCol2, etc) values ($_.Subject, $_.body, etc)"
} | out-file "outfile.sql" -Append
I don't know the answer, but, if you take a look at Google email uploader (open source), they do the reading part...
I would write an automation client in C# to iterate through the outlook emails, then upload each one to your database. None of this stuff is rocket science. The automation client requires Outlook to be installed and running on the machine. In other words, this approach does not involve just "reading" the PST ; automation implies the Outlook app is actually running and your code is asking the app to open the emails individually. (You need not display all the UI as you do this).
Here's a Q on how to read a PST file by automating outlook using C#. Starting with that, you then need to add the MySQL update stuff, and some good error handling. Be sure to test thoroughly before deleting the files from Outlook. If you choose to not delete, be sure to have a good indexing approach to insure idempotence.
A solution I just stumbled across is:
libpst
Obviously there is still some plumbing in processing one of the converted formats into SQL, but if importing into Outlook and then exporting as CSV is not an option, libpst would be a good alternative.
Been there done that :)
Yea libpst + a 6 pack of coors light is your solution here :)
like Cheeso said, not rocket science. Dump in a big table, but what to do with the attachments is where I struggled a little. First iteration ended up dumping on disk and columnize the path to "stuff".
Iteration 2, set up a small Hadoop instance and loaded the whole shebang in Hbase. I had 600Gb of emails... little OCD but works great to this day :)
I often have data in Excel or text that I need to get into SqlServer. I can use ODBC to query the Excel file and I can parse the text file. What I want though is some tool that will just grab the data and put it into tables with little / no effort. Does anyone know of such a tool?
Have you tried the SQL Server Import/Export Wizard ?
In SQL Server Management Studio, right-click your Database Name, and select Tasks menu, Import Data. For Data Source, select Microsoft Excel, browse to the .XLS...
If you are using Sql Server look at Integration Services (SSIS).
You can also take a look at parse-o-matic
Use DTS or SSIS depending on which version of SQL Server you have. There is an import wizard which can get you started, but data imports are rarely simple and usually involve some sort of data cleanup so that your incoming data is acceptable to the table where you intend to store it. Excel data, in my experience, is usually particularly bad inthis respect becasue it often isn't stored properly in Excel to begin with.
I haven't seen commercial tools that do this. I create this kind of tools at work all the time, and the data validation is not trivial. This just makes sure that you don't have bad data making it into your database.
I found that for simple data conversion needs something like FileHelpers is pretty good. It still needs programming though. This framework is fairly easy to use, and somebody with a little bit of experience could bang something out for you.
On further thought, you can use the SQL Server bcp utility to upload the contents of a text file. This is a command-line utility and has a lot of switches. I would suggest you experiment on a test table before you use this in a production table.
It's been a while since I used it, so I can't remember if you can directly use an Excel spreadsheet. Text files are always the easiest to deal with in any case.
Seems like it'd be pretty easy to write a script that reads the text file, and converts it to "INSERT * into TABLE" Sql statements. I suspect this has already been done, but a simple implementation would be less than 100 lines of code in your favorite scripting language.
Hey, Google says SQLServer comes with such a tool, BULK INSERT:
I have a .NET webforms front end that allows admin users to upload two .xls files for offline processing. As these files will be used for validation (and aggregation) I store these in an image field in a table.
My ultimate goal is to create an SSIS package that will process these files offline. Does anyone know how to use SSIS to read a blob from a table into its native (in this case .xls) format for use in a Data Flow task?
In my (admittedly limited) experience with SSIS, it is quite good at rapidly getting something up and running, but frusteratingly limited in getting something that "feels" like the most elegant, efficient solution to a programmer.
Since the Excel Source Editor seems to take only files as input, you need to give it a file or reimplement its functionality in code that can take a blob. I understand that this is unsatisfying, but in the end, this is a time saving tool.