In Matlab, I'm pulling in data from a mySQL database using a statement similar to:
SELECT PrimaryKeyVar, Var1, MyDate, Var2, Var3 FROM MyDatabase.MyTable ORDER BY PrimaryKeyVar DESC LIMIT 4
Among the 4 values returned are some NULLs. Unfortunately, these are imported into Matlab as 'null' rather than NaN (in other words, Matlab treats the mySQL NULLs as strings). Is there a way to import the NULLs as NaNs?
I was thinking of including a statement like ...IF(MyDate IS NULL, "????", MyDate) AS MyDate... (where the "????" would hold some kind of identifier for the NULLs) but I'm not sure if that can work.
You can control the behaviour of Database Toolbox on null data via preferences. Open the MATLAB Preferences dialog via File->Preferences, and navigate in the left panel to the section for Database Toolbox. Specify the behaviour you'd like in the section Null Data Handling.
Alternatively, you can control the same preferences programatically with the command setdbprefs. You may need to set the values of the preferences NullNumberRead, NullNumberWrite, NullStringRead and NullStringWrite. Type doc setdbprefs for more information.
Related
I've been tasked with converting FoxPro databases to MySQL and I need to know how to export the structure/indexes of a FoxPro database to Excel. Is this possible to export that type of information from FoxPro?
I know there are tools out there that do this kind of conversion for you but that has been rejected due to our budget. We were hoping to create a specialized conversion script that will automatically convert all our containers and dbfs.
Thank you in advance.
If you look at the download area at Leafe.com there are various free tools to migrate data from VFP into MySQL.
There is a data upload program, and a couple of tools to create MySQL CREATE TABLE scripts from the currently selected alias in Visual FoxPro.
Alternatively if you want to pursue the Excel route manually then ...
If you have a table MYTABLE.DBF with the following structure:
Structure for table: C:\TEMP\MYTABLE.DBF
Number of data records: 0
Date of last update: 01/05/2014
Code Page: 1252
Field Field Name Type Width Dec Index Collate Nulls Next Step
1 FIRSTNAME Character 20 No
2 LASTNAME Character 20 No
3 AGE Numeric 3 No
4 ID Integer (AutoInc) 4 Asc Machine No 1 1
** Total ** 48
Then you can dump the structure to another DBF via the VFP Command Window like this:
cd \temp
use mytable
copy structure extended to mytablestruct.dbf
You can then open the table that contains structure info and dump it to XLS format:
use mytablestruct
copy to struct.xls type xl5
In Excel that will look like:
With regard to indexes you would have to code a small routine like this:
Create Cursor indexinfo (idxname C(254), idxtype c(254), idxkey c(254), ;
idxfilter c(254), idxorder c(254), idxcoll c(254))
Use mytable In 0
Select mytable
lnTags = ATagInfo(laTemp)
For i = 1 to lnTags
Insert into indexinfo values (laTemp[i, 1], laTemp[i, 2], laTemp[i, 3], laTemp[i, 4], laTemp[i, 5], laTemp[i, 6])
EndFor
Select indexinfo
Copy To indexinfo.xls type xl5
Opening the resultant indexinfo.xls:
You can do it from FoxPro, and there is no need to export the info to Excel, Foxpro is capabale of recreating your databases/tables/indexes in MySQL, and upload the records.
I have developed a tool that can upload any FoxPro table to MySQL, just using FoxPro commands.
Check gendbc.prg in the tools folder and adapt it to your needs.
You will have to do some field type conversions for MySQL. Also if you are going to upload your data, there are some caveats with dates/datetimes:
Replace empty VFP date fields with '0000-00-00' in MySQL, and '0000-00-00 00:00:00' for empty datetimes.
Some useful commands are AFIELDS, ATAGINFO
All good points... Additionally with VFP, you can do with the menu at "Tools" --> "Wizards" --> "Upsizing". You will need to make a connection to the database and it will walk you through most of the stuff.
You can upsize an entire database, or just individual tables during the wizard process.
I have this union statement when I try to take parameters from a form and pass it to a union select statement it says too many parameters. This is using MS ACCESS.
SELECT Statement FROM table 1 where Date = Between [Forms]![DateIN]![StartDate]
UNION
SELECT Statement FROM table 2 where Date = Between [Forms]![DateIN]![StartDate]
This is the first time I am using windows DB applications to do Database apps. I am Linux type of person and always use MySQL for my projects but for this one have to use MS Access.
Is there anther way to pass parameters to UNION Statement because this method of defining values in a form can work on Single SELECT statements. But I don't know why this problem exist.
Between "Determines whether the value of an expression falls within a specified range of values" like this ...
expr [Not] Between value1 And value2
But your query only gives it one value ... Between [Forms]![DateIN]![StartDate]
So you need to add And plus another date value ...
Between [Forms]![DateIN]![StartDate] And some_other_date
Also Date is a reserved word. If you're using it as a field name, enclose it in brackets to avoid confusing the db engine: [Date]
If practical, rename the field to avoid similar problems in the future.
And as Gord pointed out, you must also bracket table names which include a space. The same applies to field names.
Still getting problems when using this method of calling the values or dates from the form to be used on the UNION statement. Here is the actual query that I am trying to use.
I don't want to recreate the wheel but I was thinking that if the Date() can be used with between Date() and Date()-6 to represent a 7 days range then I might have to right a module that takes the values from the for and then returns the values that way I can do something like Sdate() and Edate() then this can be used with Between Sdate() and Edate().
I have not tried this yet but this can be my last option I don't even know if it will work but it is worth a try. But before i do that i want to try all the resources that Access can help me make my life easy such as its OO Stuff it has for helping DB programmers.
SELECT
"Expenditure" as [TransactionType], *
FROM
Expenditures
WHERE
(((Expenditures.DateofExpe) Between [Forms]!Form1![Text0] and [Forms]![Form1]![Text11]))
UNION
SELECT
"Income" as [TransactionType], *
FROM
Income
WHERE
(((Income.DateofIncom) Between [Forms]!Form1![Text0] and [Forms]![Form1]![Text11] ));
Access VBA has great power but I don't want to use it as of yet as it will be hard to modify changes for a user that does not know how to program. trying to keep this DB app simple as possible for a dumb user to fully operate.
Any comments is much appreciated.
I've searched and can't seem to find quite what I'm looking for.
I'm running a PL/SQL script in Oracle, and attempting to insert records into a table in MySQL via database link using MySQL ODBC 5.2 Unicode Driver.
The link works fine, I can do complex queries in Oracle using it, and do various inserts and updates on records there.
Where it fails is in trying to insert a record into a MySQL table that has a column of type bit(1).
It is basically a cursor for loop, with the insert statement looking something like:
INSERT INTO "app_user"#mobileapi (USERNAME, VERSION, ACCOUNT_EXPIRED, ACCOUNT_LOCKED, PASSWD, PASSWORD_EXPIRED)
VALUES (CU_rec.USERNAME, CU_rec.VERSION, CU_rec.ACCOUNT_EXPIRED, CU_rec.ACCOUNT_LOCKED, CU_rec.PASSWD, CU_rec.PASSWORD_EXPIRED)
Some of the target columns, like ACCOUNT_EXPIRED, ACCOUNT_LOCKED, etc. are the bit(1) columns in MySQL. Given that I can convert the data types in the cursor CU_rec to pretty much anything I want in Oracle, how can I get them inserted into the target? I've tried everything I can think of, and I just keep getting:
Error report:
ORA-28500: connection from ORACLE to a non-Oracle system returned this message:
[MySQL][ODBC 5.2(w) Driver][mysqld-5.6.10]Data too long for column 'ACCOUNT_EXPIRED' at row 1 {HY000,NativeErr = 1406}
ORA-02063: preceding 2 lines from MOBILEAPI
ORA-06512: at line 44
28500. 00000 - "connection from ORACLE to a non-Oracle system returned this message:"
*Cause: The cause is explained in the forwarded message.
*Action: See the non-Oracle system's documentation of the forwarded
message.
Any help at all would be greatly appreciated.
Your problem is Oracle's default datatype conversion over ODBC; according to their own documentation they convert SQL_BINARY to a raw. Although not directly related, Oracle's comparison of MySQL and Oracle within SQL Developer also alludes to the fact that the automatic conversion from a MySQL bit is to an Oracle raw.
Extremely confusingly, MySQL's documentation indicates that a bit is converted to a SQL_BIT or a SQL_CHAR, which implies that it may work in the other direction1.
According to Microsoft's ODBC docs you should, theoretically, be able to use the CONVERT() function to transform this into a character, which should, theoretically, be translatable by MySQL.
insert into some_table#some_db (bit_col)
values( {fn convert(some_col, SQL_CHAR)} );
Failing that there's another couple of options, but it does depend on what you're attempting to insert into the MySQL database from Oracle and what the datatype is in Oracle. For instance you could use the Oracle CAST() function to convert between datatypes. For instance, the following would convert an integer to a binary double.
select cast(1 as binary_double) from dual
Unfortunately, you can't cast an integer to a raw, only a character or a rowid, so in order to convert to a raw you'd have to do the following:
select cast(to_char(1) as raw(1)) from dual
I've no idea whether MySQL will accept this but with some testing you should be able to work it out.
1. For clarity, I've never tried it in either direction.
Hah! I found a solution. Dropping it here in case it helps someone else. It's not pretty, but it works.
I used the old EXECUTE IMMEDIATE trick.
Basically, I created a variable sql_stmt varchar2(4000) and wrote code like:
sql_stmt := 'insert into "app_user"#mobileapi (USERNAME, VERSION, ACCOUNT_EXPIRED, ACCOUNT_LOCKED, CIPHER_PASSPHRASE, ENABLED, PASSWD, PASSWORD_EXPIRED)
values ('''||CU_rec.USERNAME||'','||CU_rec.VERSION||', '||CU_rec.ACCOUNT_EXPIRED||', '||CU_rec.ACCOUNT_LOCKED||', '''||CU_rec.CIPHER_PASSPHRASE||''', '||
CU_rec.ENABLED||', '''||CU_rec.PASSWD||''', '||CU_rec.PASSWORD_EXPIRED||')';
EXECUTE IMMEDIATE sql_stmt;
Something like that anyway (the quotes might not line up, as I hacked this a bit from the actual code). Looking at the contents of sql_stmt, I get:
insert into "app_user"#mobileapi (USERNAME, VERSION, ACCOUNT_EXPIRED, ACCOUNT_LOCKED, CIPHER_PASSPHRASE, ENABLED, PASSWD,PASSWORD_EXPIRED)
values ('user#email.com', 0, 0, 0, 'asdfastrwaebawavgansdhnsgjsjsh', 1, 'awercbcakwjerhcawuerawieubkahbewvkruh', 0)
The EXECUTE IMMEDIATE completes, and checking the target table, the values are there.
Possibly a crappy solution, but better than nothing.
We've just upgraded from Access 2003 to Access 2010 and string comparisons are failing with an invalid procedure call error when default conditions are used. I’ve recreated two presumably related problems in a new Access 2007 format database containing only the default table, a query with the SQL below and a module containing only the code below, so I seriously doubt that this is a corruption issue.
First the following sub fails on the If Then line with Run-time error 5: Invalid procedure call or argument
Option Compare Database
Option Explicit
Sub checkStrCmp()
Dim str As String
str = "s"
If str = "s" Then
MsgBox "works"
End If
End Sub
If I change Option Compare Database to Option Compare Text the sub works as expected, but this seems like a bad idea as I may want to preform text as well as numeric comparisons inside a single sub.
I’m also getting “Invalid procedure call” errors in string comparison functions inside of SQL. The Replace function is requiring the supposedly optional compare parameter.
Select replace("foo-bar-baz", "-", "|", 1,-1);
Generates the “Invalid procedure call” error
Setting the compare parameter to any of the available values (0 -3) works as expected:
SELECT replace("foo-bar-baz", "-", "|", 1,-1, 0);
produces “foo|bar|baz”
Has anyone else seen this? Is there a setting that needs to be tweaked? Any other ideas outside of “Database corruption” which is all I’ve been able to find via Google.
TIA
apoligies for the sloppy code blocks I can't for the life of me get them to work right.
UPDATE: I should have mentioned that I'm running XP Pro sp3.
The problem seems limited to databases I create on my box. When I opened the test database I created on my box from other workstations on our network I saw the issue, but was then unable to recreate it when creating a new database as described above on those workstations. The databases I created on the two other workstations (same OS and MS Office versions installed) also worked correctly when opened on my machine. I was also unable to recreate the issue when I inserted new modules in those DBs from my machine.
In short the problem seems to only exist on databases created on my machine (and in old 2003 format databases I've converted to 2007 format on my machine). My best guess is that my install is hosed but I’d like to have some idea of how and why before I approach IT with a request to reinstall Office. I’d also like to rule out a conflict with other software on my box.
Your code modules do not all need to share the same Option Compare setting. So you could place those procedures which should use text comparisons in a module which has Option Compare Text in its Declarations section.
However, I don't understand your statement, "I may want to preform text as well as numeric comparisons inside a single sub." According to Access' help topic, the Option Compare Statement is "Used at module level to declare the default comparison method to use when string data is compared". In other words, Option Compare has no effect on the comparisons of numeric values.
Edit: Since the problem is limited to Option Compare Database for database files created on only one machine, I'll suggest you check Access' "New database sort order" setting on that machine. Change it to a choice which starts with "General" if it is set to anything else. Then create a new database and see whether you still have the problem.
The reason for this suggestion is that Option Compare Database tells Access to use the database's codepage setting for sorting. And "New database sort order" can set the codepage to the one which never gives me such troubles. However, my understanding of codepage details is pretty shallow; I never change it and don't know what the consequences of other settings would be.
Is there any way to directly access the stemmer used in the FORMSOF() option of a CONTAINS Full Text Search query so that it returns the stems/inflections of an input word, not just those derivations that exist in a search column.
For example, the query
SELECT * FROM dbo.MyDB WHERE contains(CHAR_COL,'FORMSOF(INFLECTIONAL, prettier)')
returns the stem "pretty" and other inflections such as "prettiest" if they exists in the CHAR_COL column. What I want is to call the FORMSOF() function directly without referencing a column at all. Any chance?
EDIT:
The query that met my needs ended up being
SELECT * FROM
(SELECT ROW_NUMBER() OVER (PARTITION BY group_ID ORDER BY GROUP_ID) ord, display_term
from sys.dm_fts_parser('FORMSOF( FREETEXT, running) and FORMSOF(FREETEXT, jumping)', 1033, null, 1)) a
WHERE ord=1
Requires membership in the sysadmin
fixed server role and access rights to
the specified stoplist.
No. You can not do this. You can't get an access to stemmer directly.
You can get an idea of how it works by looking into Solr source code. But it might (and I guess will) be different from the one implemented in MS SQL FT.
UPDATE: It turns out that in SQL Server 2008 R2 you can do something quite close to what you want. A special table-valued UDF was added:
sys.dm_fts_parser('query_string', lcid, stoplist_id, accent_sensitivity)
it allows you to get a tokenization result (i.e. the result after applying word breaking, thesaurus and stop list application). So in case you feed it 'FORMSOF(....)' it will give you the result you want (well, you will have to process result set anyway). Here's corresponding article in MSDN.