Import Foxpro tables into SQL Server - sql-server-2008

We have 800 different .dbf files and these need to load into SQL Server with their file name as the new table name, so file1.dbf has to be loaded into SQL Server into table file1.
Like this, we need to load all 800 Foxpro tables into SQL Server. Does anyone have an idea for this, or a script? Any help is highly appreciated.

There are multiple solutions to the problem. One is to use the upsizing wizard that ships with VFP. I only tried the original version and it was not good at all. I didn't use it since then. You may try uploading a test database with a single table that has, say a million rows in it, just to see if using that would be feasible to use (a million rows shouldn't take more than a minute).
What I did was to create a "generator" that would create the SQL server tables in the mapping I wanted (ie: memo to varchar or varbinary MAX, char to varchar etc). Then using a C# based activex code I wrote, I load the tables - multiple tables at a time (other ways of loading the tables were extremely slow). Since then that code is used to create SQL server tables and\or transfer exiting customers' data to SQL server.
Yet another effective way would be, create a linked server to VFP using VFPOLEDB and then use OpenQuery to get tables' structure and data:
select * into [TableName]
from OpenQuery(vfpserver, 'select * from TableName ...')
This one is fast too and allows you to use VFP specific functions inside the query, however resulting field types might not be as you like.

Below is a solution that is written in FoxPro 9. You will probably need to modify a bit as I only handled 3 data types. You will also have to look out for SQL reserved words as field names.
SET SAFETY OFF
CLOSE ALL
CLEAR ALL
CLEAR
SET DIRE TO "C:\temp"
** house keeping
RUN del *.fxp
RUN del *.CDX
RUN del *.bak
RUN del *.err
RUN del *.txt
oPrgDir = SYS(5)+SYS(2003) && Program Directory
oPath = "C:\temp\pathtodbfs" && location of dbfs
CREATE TABLE dbfstruct (fldno N(7,0), fldnm c(16), fldtype c(20), fldlen N(5,0), fldpoint N(7,0)) && dbf structure table
STORE SQLSTRINGCONNECT("DRIVER={MySQL ODBC 3.51 Driver};SERVER=localhost;DATABASE=testdbf;UID=root;PWD=root; OPTION=3") TO oConn && SQL connection
SET DIRE TO (m.oPath)
STORE ADIR(aFL, "*.dbf") TO iFL && getting list of dbfs
SET DIRE TO (m.oPrgDir)
FOR i = 1 TO iFL
IF AT("dbfstruct.dbf", LOWER(aFL(i,1))) = 0 THEN
USE oPath + "\" + aFL(i,1)
LIST STRUCTURE TO FILE "struct.txt" && output dbf structure to text file"
SET DIRE TO (m.oPrgDir)
USE dbfstruct
ZAP
APPEND FROM "struct.txt" TYPE SDF
DELETE FROM dbfstruct WHERE fldno = 0 && removing non esential text
PACK
CLEAR
DELETE FILE "struct.txt"
SET DIRE TO (m.oPrgDir)
=SQLEXEC(oConn, "DROP TABLE IF EXISTS testdbf." + STRTRAN(LOWER(aFL(i,1)),".dbf", "")) && needed to remove tables already created when I was testing
sSQL = "CREATE TABLE testdbf." + STRTRAN(LOWER(aFL(i,1)),".dbf", "") + " ("
SELECT dbfstruct
GOTO TOP
DO WHILE NOT EOF()
#1,1 SAY "CREATING QUERY: " + aFL(i,1)
sSQL = sSQL + ALLTRIM(LOWER(dbfstruct.fldnm)) + " "
* You may have to add below depending on the field types of your DBFS
DO CASE
CASE ALLTRIM(dbfstruct.fldtype) == "Character"
sSQL = sSQL + "VARCHAR(" + ALLTRIM(STR(dbfstruct.fldlen)) + "),"
CASE ALLTRIM(dbfstruct.fldtype) == "Numeric" AND dbfstruct.fldpoint = 0
sSQL = sSQL + "INT(" + ALLTRIM(STR(dbfstruct.fldlen)) + "),"
CASE ALLTRIM(dbfstruct.fldtype) == "Numeric" AND dbfstruct.fldpoint > 0
sSQL = sSQL + "DECIMAL(" + ALLTRIM(STR(dbfstruct.fldlen)) + "),"
OTHERWISE
=MESSAGEBOX("Unhandled Field Type: " + ALLTRIM(dbfstruct.fldtype) ,0,"ERROR")
CANCEL
ENDCASE
SELECT dbfstruct
SKIP
ENDDO
sSQL = SUBSTR(sSQL, 1, LEN(sSQL)-1) + ")"
STORE SQLEXEC(oConn, sSQL) TO iSQL
IF iSQL < 0 THEN
CLEAR
?sSQL
STORE FCREATE("sqlerror.txt") TO gnOut && SQL of query in case it errors
=FPUTS(gnOut, sSQL)
=FCLOSE(gnOut)
=MESSAGEBOX("Error creating table on MySQL",0,"ERROR")
CANCEL
ENDIF
CLOSE DATABASES
ENDIF
ENDFOR
=SQLDISCONNECT(oConn)
SET DIRE TO (m.oPrgDir)
SET SAFETY ON

Related

Set default value or ignore Append Query Field that does not exist in MS-Access

I have a series of Excel Tools that work with an Access database (which is really the front end to a series of SharePoint lists). Each Excel Tool has an export function that creates a separate file (CSV) containing data to bring into the database. In the database, the CSV comes in as a local table, and an append query is run to add the contents of the local table to the SharePoint list. I am not fond of the architecture, but due to the robustness of my company's IT Security protocols, this is what I have to do.
There is a list of columns that COULD exist in the CSV, but whether they exist or not depends on the data entered into the workbook. For example, here are all the possible column headings:
kWh Savings
kw Savings
Natural Gas Savings
Water Savings
Fuel Oil Savings
Propane Savings
Depending on the customer/building/end use, some of the fields might not exist. For example, if this is a lighting calculator, no water savings will exist, and therefore that columns will not exist. Since the field does not exist, the append query throws a dialog box to ask me the parameter value. What I'm trying to accomplish is for it not to do this. I don't have a preference whether it just ignores the field, or if it defaults to 0. Any thoughts? Here are my queries that make the whole thing work:
[AppendFilter]
SELECT csv.*
FROM csv LEFT JOIN sharepoint ON (csv.FIM_Unique = sharepoint.FIM_Unique) AND (csv.[Building ID] = sharepoint.Building)
WHERE (((sharepoint.FIM_Unique) Is Null) AND ((sharepoint.Building) Is Null) AND ((csv.FIM_Unique) Is Not Null));
[AppendData]
INSERT INTO sharepoint ( FIM_Unique, FIM_Designation, [FIM Description], Safety_Factor, Building, [kWh Savings], [kW Savings], [Natural Gas Savings], [Water Savings], [Fuel Oil Savings], [Propane Savings] )
SELECT [AppendFilter].FIM_Unique, csv.[FIM Designation], csv.[FIM Description], csv.[Safety Factor], [AppendFilter].[Building ID], [AppendFilter].[kWh Savings], [AppendFilter].[kW Savings], [AppendFilter].[Natural Gas Savings], [AppendFilter].[Water Savings], [AppendFilter].[Fuel Oil Savings], [AppendFilter].[Propane Savings]
FROM [AppendFilter]. INNER JOIN csv ON [AppendFilter].FIM_Unique = csv.FIM_Unique
WHERE ((([AppendFilter].FIM_Unique) Is Not Null) AND (([AppendFilter].[Building ID]) Is Not Null));
I have tried DoCmd.SetWarnings False when running the query, but these warnings appear immune to that!
Could use VBA to dynamically build and execute SQL action. One way is with DAO TableDefs to iterate through table fields and build a string of field names and use string variable to build SQL statement. I don't think need to join CSV with AppendFilter. For one thing, doing so results in a SELECT query with duplicate field names and that will cause confusion unless fields are prefixed with table name.
However, gets complicated if field names are not same in source and destination tables. I see some variation with underscores and Building ID as opposed to Building. Since there are only 3 fields with variation, build string variable with conditional code to check for particular field names and make adjustment as appropriate.
Dim db As DAO.Database, td As DAO.TableDef, fd As DAO.Field, strD As String, strF As String
Set db = CurrentDb
Set td = db.TableDefs("csv")
For Each fd In td.Fields
strF = fd.Name
If strF = "FIM Designation" Or strF = "Safety Factor" Then strF = Replace(strF, " ", "_")
If strF = "Building ID" Then strF = "Building"
strD = strD & "[" & strF & "],"
Next
strD = Left(strD, Len(strD) - 1)
db.Execute "INSERT INTO sharepoint(" & strD & ") " & _
"SELECT * FROM csv " & _
"WHERE FIM_Unique IN(SELECT FIM_Unique FROM AppendFilter)"
I strongly advise not to use spaces nor punctuation/special characters in naming convention. Underscore is only exception but I never use that either.

Parameter inside identifier

I'm trying to write a add query that will change depending on the parameter. I have several queries:
LastK1StatDate
LastK2StatDate
.
.
LastK15StatDate
LastK16StatDate
My criteria should change depending on the value entered for the parameter "qryKioskNum" when the query is run.
Currently my criteria is this:
>Max("[LastK" & [qryKioskNum] & "StatDate]![K" & [qryKioskNum] & "LastDate]")
qryKioskNum is type Short Text
It keeps giving me the error "The expression is typed incorrectly, or is too complex to be evaluated."
Here is the complete SQL statement for this query:
PARAMETERS qryKioskNum Short;
INSERT INTO K1DispRejStat ( K1StatDate, K1BillCount1, K1BillCount2,
K1BillCount3, K1BillCount4, K1BillCount5, K1BillCount6, K1BillRej1,
K1BillRej2, K1BillRej3, K1BillRej4, K1BillRej5, K1BillRej6 )
SELECT DateValue([responseFrames]![dispDateTime]) AS [Date],
Sum(responseFrames.billCount1) AS SumOfbillCount1,
Sum(responseFrames.billCount2) AS SumOfbillCount2,
Sum(responseFrames.billCount3) AS SumOfbillCount3,
Sum(responseFrames.billCount4) AS SumOfbillCount4,
Sum(responseFrames.billCount5) AS SumOfbillCount5,
Sum(responseFrames.billCount6) AS SumOfbillCount6,
Sum(responseFrames.BillRej1) AS SumOfBillRej1, Sum(responseFrames.BillRej2)
AS SumOfBillRej2, Sum(responseFrames.BillRej3) AS SumOfBillRej3,
Sum(responseFrames.BillRej4) AS SumOfBillRej4, Sum(responseFrames.billRej5)
AS SumOfbillRej5, Sum(responseFrames.billRej6) AS SumOfbillRej6
FROM responseFrames, LastK1StatDate
WHERE (((responseFrames.kioskID)="K1"))
GROUP BY DateValue([responseFrames]![dispDateTime])
HAVING (((DateValue([responseFrames]![dispDateTime]))>Max("[LastK" &
[qryKioskNum] & "StatDate]![K1LastDate]")))
ORDER BY DateValue([responseFrames]![dispDateTime]);
currently everything is set to "K1" but I would like all reference to K1 to be dynamic
I think it is just a syntax issue but can't find how exactly this should be typed out.
Any help is great. Thanks!
*edited for clarity
In msaccess, create a PassThru query (because it retains the multi-line nice format).
Create // QueryDesign // Close // rightClick // SQLSpecific // PassThru
Paste in the following sql.
INSERT INTO kxxdisprejstat
(kxxstatdate,
kxxbillcount1,
kxxbillcount2,
kxxbillcount3,
kxxbillcount4,
kxxbillcount5,
kxxbillcount6,
kxxbillrej1,
kxxbillrej2,
kxxbillrej3,
kxxbillrej4,
kxxbillrej5,
kxxbillrej6)
SELECT Datevalue([responseframes] ! [dispdatetime]) AS [Date],
SUM(responseframes.billcount1) AS SumOfbillCount1,
SUM(responseframes.billcount2) AS SumOfbillCount2,
SUM(responseframes.billcount3) AS SumOfbillCount3,
SUM(responseframes.billcount4) AS SumOfbillCount4,
SUM(responseframes.billcount5) AS SumOfbillCount5,
SUM(responseframes.billcount6) AS SumOfbillCount6,
SUM(responseframes.billrej1) AS SumOfBillRej1,
SUM(responseframes.billrej2) AS SumOfBillRej2,
SUM(responseframes.billrej3) AS SumOfBillRej3,
SUM(responseframes.billrej4) AS SumOfBillRej4,
SUM(responseframes.billrej5) AS SumOfbillRej5,
SUM(responseframes.billrej6) AS SumOfbillRej6
FROM responseframes,
lastkxxstatdate
WHERE (( ( responseframes.kioskid ) = "kxx" ))
GROUP BY Datevalue([responseframes] ! [dispdatetime])
HAVING (( ( Datevalue([responseframes] ! [dispdatetime]) )
> Max([lastkxxstatdate]![kxxlastdate]) ))
ORDER BY Datevalue([responseframes] ! [dispdatetime]);
Name it kxxInsert (or some such, using kxx to say that it is generalized).
Then add this to the program
Sub getKxx()
Dim qrykiosknum As Integer ' temp here to have something
qrykiosknum = 3 ' temp here for an example
Dim kxxSQL As String, strSQL As String
kxxSQL = CurrentDb.QueryDefs("kxxInsert").SQL
strSQL = Replace(kxxSQL, "kxx", "k" & qrykiosknum)
'MsgBox (strSQL) ' it is too big to see all of it
Debug.Print strSQL
' then run strSQL
End Sub
Having dynamic tablename in MSAccess or MSSQLServer is possible when the Replace is done before executing the SQL.
I doubt you can make this work by using a query with a parameter. You are much better off using VBA. Use InputBox to get the variable portion of the query and DoCmd.RunSQL to run the query.

How to create MySQL database programmatically using matlab

I am doing some data analytics using MySQL and Matlab. My program works fine if there is already an existing database present. But it doesn't work when there is not database for which I am trying to create connection. Now, what I want to do is to create a database with a fixed name if there is no database present in that name. I searched all over the internet and haven't found any option for that. May be I am missing something.
Additionally, I would like to create a table on the newly created database and store some random data on them. I can do this part. But I am stucked on the first part which is creating database programmatically using matlab.
Please note that, I have to use only matlab for this project. Any kind cooperation will be greatly appreciated.
Update
The code example is given below -
%findIfFeederExists Summary of this function goes here
% finds whether there is no. of feeders are empty or not
% Detailed explanation goes here
%Set preferences with setdbprefs.
setdbprefs('DataReturnFormat', 'dataset');
setdbprefs('NullNumberRead', 'NaN');
setdbprefs('NullStringRead', 'null');
%Make connection to database. Note that the password has been omitted.
%Using ODBC driver.
conn = database('wisedb', 'root', '');
conn.Message;
%Read data from database.
sqlQuery = 'SELECT * FROM joined_table';
%sqlQuery = 'SELECT * FROM joined_table where joined_table.`N. of Feeder` > 0';
curs = exec(conn, sqlQuery);
curs = fetch(curs);
dbMatrix = curs.data;
[row_count, ~] = size(dbMatrix);
if (row_count >= id)
val = dbMatrix(id, 3);
disp(val);
if (val.N0x2EOfFeeder > 0)
Str = strcat('Feeder is present on the id : ', num2str(id));
disp(Str);
disp(dbMatrix(id, 1:end));
else
Str = strcat('Feeder is NOT present on the id : ', num2str(id));
disp(Str);
end
else
Str = strcat('No row found for id : ', num2str(id));
disp(Str);
end
% = exec(conn,'SELECT * FROM inventoryTable');
close(curs);
%Assign data to output variable
%Close database connection.
close(conn);
%Clear variables
clear curs conn
end
You can see I can connect to an existing database using ODBC. But I am not sure how I can create a new database. What can be done for this?

Insert Data to MYSQL using Foxpro

In FoxPro using native table, I usually do this when inserting new Data.
Sele Table
If Seek(lcIndex)
Update Record
Else
Insert New Record
EndIf
If I will use MYSQL as my DataBase, what is the best and fastest way to
do this in FoxPro code using SPT? I will be updating a large number of records.
Up to 80,000 transactions.
Thanks,
Herbert
I would only take what Jerry supplied one step further. When trying to deal with any insert, update, delete with SQL pass through, it can run into terrible debugging problems based on similar principles of SQL-injection.
What if your "myValue" field had a single quote, double quote, double hyphen (indicating comment)? You would be hosed.
Parameterize your statement such as using VFP variable references, then use "?" in your sql statement to qualify which "value" should be used. VFP properly passes. This also helps on data types, such as converting numbers into string when building the "myStatement".
Also, in VFP, you can use TEXT/ENDTEXT to simplify the readability of the commands
lcSomeStringVariable = "My Test Value"
lnANumericValue = 12.34
lnMyIDKey = 389
TEXT to lcSQLCmd NOSHOW PRETEXT 1+2+8
update [YourSchems].[YourTable]
set SomeTextField = ?lcSomeStringVariable,
SomeNumberField = ?lnANumericValue
where
YourPKColumn = ?lnMyIDKey
ENDTEXT
=sqlexec( yourHandle, lcSQLCmd, "localCursor" )
You can use SQL Pass through in your Visual Foxpro application. Take a look at the SQLCONNECT() or SQLSTRINGCONNECT() for connecting to your Database. Also look at SQLEXEC() for executing your SQL statement.
For Example:
myValue = 'Test'
myHandle = SQLCONNECT('sqlDBAddress','MyUserId','MyPassword')
myStatement = "UPDATE [MySchema].[Mytable] SET myField = '" + myValue + "' WHERE myPk = 1"
=SQLEXEC(myHandle, myStatement,"myCursor")
=SQLEXEC(myHandle, "SELECT * FROM [MySchema].[Mytable] WHERE myPk = 1","myCursor")
SELECT myCursor
BROWSE LAST NORMAL
This would be your statement string for SQLEXEC:
INSERT INTO SOMETABLE
SET KEYFIELD = ?M.KEYFIELD,
FIELD1 = ?M.FIELD1
...
FIELDN = ?M.FIELDN
ON DUPLICATE KEY UPDATE
FIELD1 = ?M.FIELD1
...
FIELDN = ?M.FIELDN
Notice that the ON DUPLICATE KEY UPDATE part does not contain the key field, otherwise it would normally be identical to the insert (or not, if you want to do something else when the record already exists)

How to write Case Sensitive Query for MS Access?

I want to know the Select Query for MS Access with case sensitive.
I have two values for VitualMonitorName as below
VCode VirtualMonitorName
Row 1 (1, 'VM1');
Row 2 (2, 'Vm1');
Here both values are different.
If I write
"SELECT VCode FROM VirtualMaster WHERE VirtualMonitorName like '" + Vm1 + "'";
It replies VCode = 1 Only.
You can use the StrComp() function with vbBinaryCompare for a case-sensitive comparison. Here is an example from the Immediate window to show how StrComp() works. See the Access help topic for more details.
? StrComp("a", "A", vbBinaryCompare)
1
? StrComp("a", "A",vbTextCompare)
0
StrComp() returns 0 if the first two arguments evaluate as equal, 1 or -1 if they are unequal, and Null if either argument is Null.
To use the function in a query, supply the vbBinaryCompare constant's value (0) rather than its name.
SELECT VCode
FROM VirtualMaster
WHERE StrComp(VirtualMonitorName, "Vm1", 0) = 0;
This approach is also available to queries from other applications if they use the newer Access Database Engine ("ACE") drivers. For example, the following C# code
string myConnectionString =
#"Driver={Microsoft Access Driver (*.mdb, *.accdb)};" +
#"Dbq=C:\Users\Public\Database1.accdb;";
using (OdbcConnection con = new OdbcConnection(myConnectionString))
{
con.Open();
using (var cmd = new OdbcCommand())
{
cmd.Connection = con;
cmd.CommandText =
"SELECT COUNT(*) AS n FROM [VirtualMaster] " +
"WHERE StrComp([VirtualMonitorName],?,?) = 0";
cmd.Parameters.AddWithValue("?", "Vm1");
cmd.Parameters.Add("?", OdbcType.Int);
var vbCompareOptions = new Dictionary<string, int>()
{
{"vbBinaryCompare", 0},
{"vbTextCompare", 1}
};
string currentOption = "";
currentOption = "vbBinaryCompare";
cmd.Parameters[1].Value = vbCompareOptions[currentOption];
Console.WriteLine(
"{0} found {1} record(s)",
currentOption,
Convert.ToInt32(cmd.ExecuteScalar()));
currentOption = "vbTextCompare";
cmd.Parameters[1].Value = vbCompareOptions[currentOption];
Console.WriteLine(
"{0} found {1} record(s)",
currentOption,
Convert.ToInt32(cmd.ExecuteScalar()));
}
}
produces
vbBinaryCompare found 1 record(s)
vbTextCompare found 2 record(s)
Check this out:
https://support.microsoft.com/kb/244693?wa=wsignin1.0
This article describes four methods of achieving a case-sensitive JOIN using the Microsoft Jet database engine. Each of these methods has advantages and disadvantages that should be weighed before choosing an implementation. The methods are:
StrComp
Case-Sensitive IISAM Driver
Hexadecimal Expansion
Binary Field
Using only built-in functions, add an additional custom column in the query design view:
location: InStr(1,[VCode],"VM1",0)
the zero parameter requests binary compare (case sensitive) when finding location of "VM1" within [VCode]
set the criteria in that column to >0 so only records with non-zero location in the vcode matching Like "*vm*" contain the exact VM1 string -
The WHERE clause looks like:
WHERE (((VirtualMaster.VCode) Like "\*vm*") AND ((InStr(1,[VCode],"VM1",0))>0));
Using at a simpler level of coding.
As a condition in a DCOUNT operation, checking on a Field (Column) that has to have the correct Case, and ignoring Blank States/Territories.
' lngcounter will count the all States
' or Territories Field ( Column) with this
' exact case value of 'Ohio'. ([ID] is an Autonumber ID field)
lngCounter = DCount("[id]", Trim(Me!tboDwellingTablename), "[State/territory],'Ohio',0) = 0")
This only does one letter:
MS-ACCESS SQL:
SELECT Asc(Left([Title],1)) AS t FROM Master WHERE (((Asc(Left([Title],1)))=105));
Title is the field you want to search
Master is the Table where Title field is located
105 Ascii code for character..
In this case only Title's that start with i not I
If you want to search for lower case "a" you would change the 105 to 97