MySQL, XML boolean values, and integers: Incorrect integer value: - mysql

Im working on grabbing xml data from ebay and putting the results into MySQL, Grabbing the data works fine, however inputting to database fails due to an incorrect integer value for a couple of the xml tags values.
The xml tag value is the word "true" (without the quotes), and this is the db sql:
CREATE TABLE ebay_categories (
CategoryID int(10) NOT NULL default '0',
CategoryLevel int(5) NOT NULL default '0',
CategoryName varchar(120) NOT NULL default '',
CategoryParentID int(10) NOT NULL default '0',
LeafCategory int(1) NOT NULL default '0',
AutoPayEnabled int(1) NOT NULL default '0',
Expired int(1) NOT NULL default '0',
IntlAutosFixedCat int(1) NOT NULL default '0',
Virtual int(1) NOT NULL default '0',
LSD int(1) NOT NULL default '0',
ORPA int(1) NOT NULL default '0',
PRIMARY KEY (CategoryID),
KEY catlevel (CategoryLevel),
KEY parent (CategoryParentID),
KEY ape (AutoPayEnabled),
KEY expired (Expired),
KEY IAFC (IntlAutosFixedCat),
KEY virtual (Virtual),
KEY lsd (LSD),
KEY orpa (ORPA),
KEY leaf (LeafCategory)
) TYPE=MyISAM;
i have tried int, tinyint, Boolean (resorts to tinyint) to no avail and still get this issue. Theres nothing wrong with the db connection as i ran a test using varchar as the int type for the LeafCategory and others and everything worked ok.
is theres something i can do without resorting to searching and replacing via regex before db insertion?
edit:
$query = "INSERT INTO `ebay_categories` (`CategoryID`, `LeafCategory`)VALUES ('$xmlCategoryID', '$xmlLeafCategory')";
if (mysqli_query($link, $query)) {
echo "Successfully inserted " . mysqli_affected_rows($link) . " row";
} else {
echo "Error occurred: " . mysqli_error($link);
}
The SQL statement unwrapped from client code is:
INSERT INTO `ebay_categories`
(`CategoryID`, `LeafCategory`)
VALUES
('$xmlCategoryID', '$xmlLeafCategory')";

The error you have is telling you at least one record in your XML has something other than a valid integer for either $xmlCategoryID or $xmlLeafCategory.
What to do?
A. You could have your error message display the offending data, something like this:
echo "Error occurred: " . mysqli_error($link);
echo "Bad data was either ##$xmlCategoryID## or ##$xmlLeafCategory##.";
Notice that I used ##$value## so you can detect empty strings in your error messages. You probably should do this.
B. You could try changing your column definitions for those columns to remove the NOT NULL declaration. If in fact one of those values is empty, this may fix your problem.
C. It's possible that you need bigint values for this information. That is, they could be very large numbers.
D. If you don't care enough about null or bad values to bother to detect them you could try this.
INSERT INTO `ebay_categories`
(`CategoryID`, `LeafCategory`)
VALUES
(CAST(IFNULL('$xmlCategoryID',-1) AS INT)
(CAST(IFNULL('$xmlLeafCategory',-1) AS INT)
This will turn null values into -1 and non-integer values into 0.
Edit: Oh, I understand now. Your leafCategory item isn't a number in XML, it's a true/false/empty value. Presumably false and empty mean the same thing.
Try this, using a SQL case statement to translate true/other to 1/0.
INSERT INTO `ebay_categories`
(`CategoryID`, `LeafCategory`)
VALUES (
'$xmlCategoryID',
CASE '$xmlLeafCategory' WHEN 'true' THEN 1 ELSE 0 END
)
Finally, danger! You need to use prepared statements, and you need to sanitize these values you're extracting from your XML file. If you don't somebody will be able to trick your software into destroying your database by feeding it a poisoned XML file. Please read up on SQL injection.

Related

OCaml - Timestamp field anomaly with ocaml-mysql

I'm having issues executing the following MySQL statement with the ocaml-mysql (latest version) library:
let dump_to_db text =
let insert = P.create db (s "INSERT INTO Temperature VALUES (?,?,?,?)") in
ignore (P.execute insert [| "NULL"; "xxx.xxx.xxx.xxx"; text ; "CURRENT_TIMESTAMP" |])
My problem is that the timestamp field is not set correctly. Apparently, whatever I write into the last field (might it be "CURRENT_TIMESTAMP" or simply "NULL"), I'm not able to get the proper outcome.
What happens is that the query get executed and what I see inside the table is a NULL-filled Timestamp field (even tho I have no idea how it can be possible because is defined as a NOT NULL column).
My table structure follows:
CREATE TABLE `Temperature` (
`ID` bigint(20) NOT NULL AUTO_INCREMENT,
`IP` varchar(100) DEFAULT NULL,
`Value` varchar(100) DEFAULT NULL,
`Timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`ID`)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=latin1
Prepared.execute takes array with values not arbitrary expressions (function calls). String value "CURRENT_TIMESTAMP" gets converted into timestamp and as it is not recognized as valid number is turned into zero.
Pass NULL properly and it will be default initialized according to CREATE TABLE :
P.execute_null insert [| None; Some "xxx.xxx.xxx.xxx"; Some text ; None |]

Database - Entry with length of strings - Structure?

i'm relaunching a website a found these entries in one of the database tables. I'm wondering what format it is and how i can easily recreate data of a form into this format:
CREATE TABLE IF NOT EXISTS `phpwcms_formresult` (
`formresult_id` int(11) NOT NULL AUTO_INCREMENT,
`formresult_pid` int(11) NOT NULL DEFAULT '0',
`formresult_createdate` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`formresult_ip` varchar(50) NOT NULL DEFAULT '',
`formresult_content` mediumtext NOT NULL,
PRIMARY KEY (`formresult_id`),
KEY `formresult_pid` (`formresult_pid`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
One entry looks like this:
(3179, 7, '2013-10-02 09:35:48', ':11', 'a:14:{s:13:"veranstaltung";s:4:"name";s:11:"Nick Habbel";s:22:"erziehungsberechtigter";s:12:"Petra Habbel".....;}');
In front of every string there is the length of the string "s:22: ...". What kind of format is this?
Any hint is much appreciated. Thanks!
An example of this value is done by the serialization of an array in php:
$arr = array("name"=>"Nick Habbel");
echo serialize($arr);
# output:
# a:1:{s:4:"name";s:11:"Nick Habbel";}
So, the application that inserts data in your table seems to serialize an array before the insertion. In your example an array (a) with a cardinality of 14 elements (a:14:) plus key/values with lengths and type "s" for strings, "i" for integers etc (s:13:"veranstaltung")

Subquery result returning comma separated values using IN clause MySQL

Friends,
two tables one table is
CREATE TABLE `vbw_push_notifications` (
`push_notification_id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'Primary key ,Auto increment field',
`push_notification_customer_ids` text NOT NULL COMMENT 'comma separated customer id list which was used for messaging/related customers/broadcasting',
`push_notification_message` varchar(500) NOT NULL COMMENT 'The notification message.(A new message from Veebow/A new message from <Merchant Name>/A new public deal <Deal Name> from <Merchant Name>/A new game deal <Deal Name> from <Merchant Name>',
`push_notification_time` datetime NOT NULL,
`push_notification_is_processed` tinyint(4) NOT NULL,
PRIMARY KEY (`push_notification_id`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=latin1 COMMENT='The comma separated customer ids of the customers who needs ';
-- ----------------------------
-- Records of vbw_push_notifications
-- ----------------------------
INSERT INTO `vbw_push_notifications` VALUES ('1', '165836,65802,65829,65837,65838', 'test test test', '2013-11-07 12:36:42', '0');
And I have another table with the following details.
CREATE TABLE `vbw_mobile_sessions` (
`mobile_session_id` int(10) NOT NULL AUTO_INCREMENT COMMENT 'The unique identifier for a mobile session',
`mobile_session_start_time` datetime DEFAULT NULL COMMENT 'The starting time # server of a mobile session',
`mobile_session_end_time` datetime DEFAULT NULL COMMENT 'The ending time # server of a mobile session',
`mobile_session_token` varchar(255) DEFAULT NULL COMMENT 'The mobile session token generated for this session',
`mobile_session_device_id` varchar(50) DEFAULT NULL COMMENT 'The device id of the device used for making the session',
`mobile_session_customer_id` int(10) DEFAULT NULL COMMENT 'The customer ID of the customer who made this session',
`mobile_session_device_type` tinyint(4) DEFAULT NULL COMMENT 'The type of device that customer uses for this session. 0 - iOS, 1 - Android',
PRIMARY KEY (`mobile_session_id`),
KEY `fk_mobile_session_customer_id` (`mobile_session_customer_id`)
) ENGINE=InnoDB AUTO_INCREMENT=677 DEFAULT CHARSET=latin1 COMMENT='This table holds the merchant account activation links creat';
I want to use a subquery like this.
SELECT DISTINCT(mobile_session_customer_id)
FROM vbw_mobile_sessions
WHERE mobile_session_end_time IS null
AND mobile_session_customer_id IN (SELECT push_notification_customer_ids FROM vbw_push_notifications WHERE push_notification_id=6) .
This query not returns zero rows. But i am getting result when i have used like this.
SELECT DISTINCT(mobile_session_customer_id)
FROM vbw_mobile_sessions
WHERE mobile_session_end_time IS null
AND mobile_session_customer_id IN ( SELECT DISTINCT(mobile_session_customer_id)
FROM vbw_mobile_sessions
WHERE mobile_session_end_time IS null
AND mobile_session_customer_id IN (65836,65802,65829,65837,65838)
I think its in a different format the subquery is returning the result . Can you please point out the mistake i have made. Many Thanks.
Your answer is obvious. String 'a,b,c,d' has nothing to do with set of values (a,b,c,d). This is not how it will work.
The correct solution is not to use delimiter-separated values in one field. You should normalize your DB structure and create linking table. Then place your values into it and build your query with using subquery, selecting from it.
Another, possible solution is to select your data (string data) from your field in application, split it by delimiter and substitute to another query then.
The subquery is returning you a varchar with a value '1,2,3' and you need a set of integers which is 1,2,3...
The engine is treating your result of the subquery as a varchar() not a set of integers.
You can go through this question, asking exactly what you need

Mysql Truncated incorrect DOUBLE value in INSERT query

I know there are lots of bugs like this around here but this query seems to be different, as it's an insert query. Here is the schema for the table card_info:
CREATE TABLE card_info (
card_id mediumint(8) unsigned NOT NULL AUTO_INCREMENT,
card_name_orig varchar(150) NOT NULL,
card_name_html varchar(150) NOT NULL,
card_name_search varchar(150) NOT NULL,
card_name_page varchar(150) NOT NULL,
card_cost varchar(50) DEFAULT NULL,
card_cost_converted tinyint(2) NOT NULL DEFAULT '0',
card_subtype varchar(75) DEFAULT NULL,
card_oracle_text_orig text,
card_oracle_text_html text,
card_power varchar(10) DEFAULT NULL,
card_toughness varchar(10) DEFAULT NULL,
card_loyalty tinyint(1) DEFAULT NULL,
PRIMARY KEY (card_id),
KEY card_name_nd (card_name_search),
KEY card_name_page (card_name_page),
KEY card_cost_converted (card_cost_converted),
KEY card_power (card_power),
KEY card_toughness (card_toughness),
KEY card_loyalty (card_loyalty),
FULLTEXT KEY card_oracle_text_orig (card_oracle_text_orig),
FULLTEXT KEY card_name_search (card_name_search)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
And here is my query:
INSERT INTO card_info (
card_name_orig, card_name_html, card_name_search, card_name_page, card_cost,
card_cost_converted, card_subtype, card_oracle_text_orig, card_oracle_text_html,
card_power, card_toughness, card_loyalty
)
SELECT DISTINCT
d.name_orig, d.name_html, d.name_search, d.name_page, d.cost,
COALESCE(d.cost_converted, 0), d.type_sub, d.oracle_text_orig,
d.oracle_text_html, d.`power`, d.toughness, d.loyalty
FROM card_info_de d
LEFT OUTER JOIN card_info i ON d.name_search = i.card_name_search
WHERE i.card_id IS NULL
AND d.edition_id = 'isd'
ORDER BY (d.collector_number + 0), d.collector_number;
If I perform this query, I'm getting this error:
1292 - Truncated incorrect DOUBLE value: '181a'
Please note that the value 181a is from the column card_info_de.collector_number and it is a VARCHAR(5) field, and that field isn't being inserted to the card_info table anyway, it's just being used in the order clause of the select query.
If I do the query starting from the SELECT only, I can see the correct results are being selected, but when I do the insert, it gives me the error above. Do note that If I remove the ORDER BY clause from the SELECT query, it inserts fine. I don't have a clue what I'm doing wrong.
Thanks in advance.
Just hit the same error myself and it is a bit misleading. I found the answer in another thread:
Error Code 1292 - Truncated incorrect DOUBLE value - Mysql
This message means you're trying to compare a number and a string in a WHERE or ON clause. Either make sure they have similar declarations, or use an explicit CAST to convert the number to a string.
If you turn off strict mode, the error should turn into a warning.
Barmar
So basically check for any of these columns being treated as the wrong data type:
d.name_search
i.card_name_search
d.edition_id
d.collector_number
Try
ORDER BY (d.collector_number + '0'), d.collector_number;
Terrible if true, but quite possible.

Mandatory field mysql

How to make sure that field is mandatory ? Here is what I mean
I have the following mysql table structure:
CREATE TABLE `new` (
`id` int(11) DEFAULT NULL,
`name` int(11) DEFAULT NULL,
`phone` int(11) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
Here is query with no data for phone
insert into new values(1, 'm', '');
But the query runs fine. What should be done so that mysql returns an error if there is no data for phone field? I can do that validation by php, but I'm curious how to do that in mysql.
Possibly setting the default value of the 'phone' column to NULL would make it fail insertion because it would end up null if you did not specify it.
Otherwise you're going to need to omit the phone column for the default to kick in, say in php you'd use empty($phone) ? null : $phone; or something along those lines.
INSERT INTO new VALUES(1,'m',NULL)
will cause error.
If you want to check whether is the phone number field is a blank string,
you can use a trigger in MySQL.
I haven't tested this, but I have a feeling the '' != null. What happens if you run
insert into new(id, name) values (1, 'test');
I bet you get an insert error...
Anyway, I think its probably better to be validating in PHP than waiting till you get to the database... inserts are expensive...
'' as the 3rd option doesnt make the value of phone null.. It is just equal to a blank string thats all.
if you want to see an error, replace '' with NULL.