Can't find my error - mysql

Hi here is a small snipit of a very large perl program I am working on, however I believe the error is around this part.
my $headd = "test,test1,test2,test3,ect";
my #headers = split(',', $headd);
CONNECT TO DATABASE STUFF THAT WORKS FINE
my %column_mapping = (
"GPS ALT" => 'GPS_ALT',
"GPS DTS" => 'GPS_DTS',
"GPS FIX" => 'GPS_FIX',
"GPS HDG" => 'GPS_HDG'
)
my $sql = sprintf 'INSERT INTO tablename ( %s ) VALUES ( %s )',
join( ',', map { $column_mapping{$_} } #$headers ),
join( ',', ('?') x scalar #$headers ); # note the list around the '?'
Since I am fairly new to perl, if I had to guess I would say its around here
join( ',', map { $column_mapping{$_} } #$headers ),
join( ',', ('?') x scalar #$headers );
I think I have the wrong data types for headers or something, but I am not entirely sure what the problem is, it just crashes around those lines.
If you happen to see something I did wrong that would be great:)

Your main error is in not adding use strict and use warnings to the top of your Perl code. These lines will find a number of problems that often sneak into Perl programs. They would have found this problem.
The problem is that you declare an array called #headers.
my #headers = split(',', $headd);
But the next time you try to use it, you are assuming that it is an array reference called $headers.
join( ',', map { $column_mapping{$_} } #$headers )
In Perl, $headers and #headers are two completely different variables with absolutely no connection between them. As you have never given $headers a value, trying to dereference it (#$headers) was never going to go well.
But you've worked that out from the comments. You've made the suggested fix (replace #$headers with #headers) and now you get a different error:
the problem is saying I have an initialized value in the join
I assume you mean "uninitialized" :-)
Sounds like the problem is on the same line that we discussed above (now with the fixed syntax):
join( ',', map { $column_mapping{$_} } #headers )
So what's happening here? Well, we are effectively translating values to new values. For each value in #headers we look it up in %column_mappings and return the associated value from the hash.
How could that give us an undefined value? Well, what happens if we look up a key in a hash that doesn't actually exist in the hash? Perl gives us the special value "undef". Which would trigger the warning that you are getting.
So my suggestion to you is that you double-check the values that you are getting in #headers (which will be the values from $headd) and make sure that all of those possible values have an associated key in %column_mappings.
One workaround would be to set a default value in the map. Something like this perhaps:
join( ',', map { $column_mapping{$_} // 'MISSING MAPPING' } #$headers )
That will almost certainly break your database interaction at some point further down the line - but at least it will be obvious what the problem is!
Update: To eliminate the problematic values before they get into #headers:
my #headers = grep { exists $column_mapping{$_} } split(',', $headd);
You'll want to move this statement after the definition of %column_mappings (for hopefully obvious reasons).

Related

"Unknown column in 'field list'", but column does exist

DROP TABLE IF EXISTS `transactions`;
CREATE TABLE `transactions` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`purchase_date` datetime DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO `transactions` (`purchase_date`) VALUES (NULL)
I've isolated my problem in this code. When I run it, I get the error:
[ERROR in query 3] Unknown column 'purchase_date' in 'field list'
Anyone an idea?
There is an unprintable character 30 (RecordSeparator) inserted between purchase_date and the ' in the INSERT statement. Just remove the text ('purchase_date') and rewrite it by hand it should be fine.
Nery niche solution when I got this error.
I had a BEFORE INSERT trigger on my table that did something with
NEW.`field_mysql_doesnt_think_exists`
and if I didn't pass that field to an insert statement then I would get
[ERROR in query 3] Unknown column 'field_mysql_doesnt_think_exists' in 'field list'
I just spent the better part of a day figuring this out. My problem was the same: invisible characters kiboshing the query and returning the "unknown column" error.
I solved it by wading back into Windows and removing the garbage using NotePad++.
How did the garbage get in there in the first place? I think it was because I made the mistake of copying some long complex queries into LibreOffice Writer (my functional specs document) instead of just bookmarking them in phpMyAdmin or saving them in a text editor. Pasting them from LibreOffice into the query window is where (I think) the garbage originated.
Once there, it persisted like Malaria. I couldn't even get rid of it by hand-retyping the whole query -- I had to put it into NotePad++ (Encoding menu) and show ANSI and the UTF8 combos and then remove the garbage by hand.
Once that was done, the query worked.
This can also happen if you paste a column name when building the table structure. Same error - but the unprintable/invisible characters are in the table structure, not the query.
I have had the same issue this morning and I didn't find my answer.
But I found my problem when I changed the single quotes around my query to double quotes. Something so small and an oversight can cause a real headache.
Unknown column x in "field list" - Code below wrapped in single quotes - Non working.
$likepost='INSERT INTO reaction(reaction_num,userreaction_id,timereacted,
streamitem_id,comment_posted_on)
VALUES ($reaction,$target,NOW(),$streamid,$comment_id)';
Code below wrapped in double quotes. working
$likepost="INSERT INTO reaction(reaction_num,userreaction_id,timereacted,
streamitem_id,comment_posted_on)
VALUES ($reaction,$target,NOW(),$streamid,$comment_id)";
This might not help anyone else, but adding this "just in case" it helps someone.
I receive large datasets as Excel CSV files and use a (WIL) script to convert the .csv file into an importable .sql file.
I had an error in my conversion script whereby these two lines did not reference the same table name (I had hard-coded the first location and forgot to update it):
* "INSERT INTO `old_table_name` (`cid`, `date`, etc etc"
* "CREATE TABLE IF NOT EXISTS `":_dbName:"` (etc etc "
I just changed the first line to also get the table name from the variable, and voila!
* "INSERT INTO `":_dbName:"` (`cid`, `date`, etc etc"
So check those two lines in your import SQL file.
Same error in a different scenario:
This also happens when you miss # symbol for a variable.
SET #myVar1=1; SELECT #myVar1; -- GOOD, correctly prints: 1
SET #myVar1=1; SELECT myVar1; -- BAD, prints: Unknown column 'myVar1' in 'field list'
when you want to work with mysql using a function like this:
function insert($table, $info_array){
// implode keys as columns by [`,`] glue string
$columns = implode("`,`", array_keys($info_array));
// implode values as sql ready values by [','] glue string
$values = implode("','", array_values($info_array));
// make query(careful about [`] for columns name and ['] for values)
$sql = "INSERT INTO ".$table." (`".$columns."`) VALUES ('".$values."');";
return $sql;
}
you should be careful about [ ` ] for table columns names and [ ' ] or [ " ] for values.
for example, I used above function this way:
try{
$db_insert_sample_user = $connection->query(insert(TABLE_PREFIX."users", [
"username" => "my_name_2",
"password" => md5("how457fty")
]));
echo '<pre>';
print_r($db_insert_sample_user);
echo '</pre>';
}catch (PDOException $exc){
echo '<pre>';
print_r($exc);
echo '</pre>';
}
the query string is this:
INSERT INTO php_pdo_users (`username`,`password`) VALUES ('my_name_2','ee04708d313adf4ff8ba321acf3eb568');
and the result was like : (for two users)
PHPMyAdmin Result
if you want functions based on prepared statements, test this : (placeholders, params and values, don't need [ ' ] or [ " ] at all!!!)
function insert_prepared(PDO $connection, $table, $info_array){
// columns
$columns = implode("`,`", array_keys($info_array));
// placeholders
$place_holders = [];
for ( $i = 0; count(array_keys($info_array)) > $i; $i++){
$place_holders[] = '?';
}
// convert placeholders to query string
$place_holders_str = implode(",", $place_holders);
$prepared_stmt = "INSERT INTO ".$table." (`".$columns."`) VALUES (".$place_holders_str.");";
// prepare statement
$stmt = $connection->prepare($prepared_stmt);
// values
$values = array_values($info_array);
// bind all params to values
for($i = 0; count($values) > $i; $i++){
$stmt->bindParam($i + 1, $values[$i]);
}
// execute and return results
return $stmt->execute();
}
after code execution this way :
try {
$db_insert_sample_user = insert_prepared(
$connection,
TABLE_PREFIX . "users",
[
"username" => "my_name_4",
"password" => md5( "HelloDolly#__3" )
]
);
} catch ( PDOException $exc ) {
echo "Failed : " . $exc->getMessage();
}
results is :
Results with insert_prepared function
I was using a mysql procedure and in the procedure parameter I used phone with an extra space instead of phone with no extra space, due to this when ever I called the function. It just throw error no such column as phone . Until I miraculously spotted it and corrected it using phpMyAdmin, Error went off.
I had this same PYMYSQL error in my Python program when I found this answer.
I probably violated a Pandas rule when I created a new dataframe ...
This created same the error above: A phantom column
inddata = dat[['ind_id','iso3c','date','data']]
dat.to_sql(name='inddata', con=engine, if_exists='append', index=False, chunksize=200)
This fixed the error:
dat2 = dat[['ind_id','iso3c','date','data']]
inddata = dat2.copy()
dat.to_sql(name='inddata', con=engine, if_exists='append', index=False, chunksize=200)
Look at the name of the table you are handling

Prepared statement fetches all the results in db, but not one I ask for

Trying to make my blog secure and learning prepared statements.
Although I set the variable, I still get all the entries from database. $escapedGet is real variable when I print it out. It's obviously a rookie mistake, but I cant seem to find an answer.
I need to get the data where postlink is $escapedGet not all the data.
$escapedGet = mysql_real_escape_string($_GET['article']);
// Create statement object
$stmt = $con->stmt_init();
// Create a prepared statement
if($stmt->prepare("SELECT `title`, `description`, `keywords` FROM `post` WHERE `postlink` = ?")) {
// Bind your variable to replace the ?
$stmt->bind_param('i', $postlink);
// Set your variable
$postlink = $escapedGet;
// Execute query
$stmt->execute();
$stmt->bind_result($articleTitle, $articleDescription, $articleKeywords);
while($stmt->fetch()) {
echo $articleTitle, $articleDescription, $articleKeywords;
}
// Close statement object
$stmt->close();
}
just tryed this: echo $escapedGet;
echo $_Get['artcile']
and got - some_other
thats the same entry that I have saved in database as postlink
tried to shande postlink to id, and then it worked. but why not with postlink tab?
When you are binding your data using 'i' modifier, it gets bound as integer.
Means string will be cast to 0 in the final statement.
But as mysql does type casting, your strings become zeroes in this query:
SELECT title FROM post WHERE postlink = 0;
try it and see - for the textual postlinks you will have all your records returned (as well as a bunch of warnings).
So, bind strings using s modifier, not i

MySQL Replace Syntax -- wrap specific piece of text

Within a table, there is a params row that has some JSON encoded data -- as such:
{"categories":211,"singleCatOrdering":"","menu-anchor_title":"","menu-anchor_css":"","menu_image":"","menu_text":1,"page_title":"","show_page_heading":0,"page_heading":"","pageclass_sfx":"","menu-meta_description":"","menu-meta_keywords":"","robots":"","secure":0}
What I need to do is wrap [] around the categories parameter - to look as such:
{"categories":[211],"singleCatOrdering":"","menu-anchor_title":"","menu-anchor_css":"","menu_image":"","menu_text":1,"page_title":"","show_page_heading":0,"page_heading":"","pageclass_sfx":"","menu-meta_description":"","menu-meta_keywords":"","robots":"","secure":0}
I have tried the following (and a bunch of other failed ones) with no-dice:
UPDATE j17_menu SET params = REPLACE(params,'"categories":%,','"categories":[%],') WHERE component_id = 10021;
Am I possibly using the wildcard option wrong? Any nudges in the right direction would be a huge help. Thanks!
This one will work independent of the context
UPDATE j17_menu
SET params=CONCAT (
SUBSTR(params,1,LOCATE('"categories":',params)),
'"categories":[',
substr(
params,
LOCATE('"categories":',params)+13,
LOCATE(',',params,LOCATE('"categories":',params))-LOCATE('"categories":',params)-13
),
']',
substr(params,LOCATE(',',params,LOCATE('"categories":',params)))
)
WHERE component_id = 10021;

JPA2 Criteria API .as(String.class) casting to char(1) - How do I work around this?

Using the criteria api, I have a query which does something like:
CriteriaBuilder cb = entityManager.getCriteriaBuilder();
CriteriaQuery<SourcePath> pathQuery = cb.createQuery(SourcePath.class);
Root<SourcePath> pathRoot = pathQuery.from(SourcePath.class);
pathQuery.where(cb.equal(cb.literal(0x00010002).as(String.class), (pathRoot.get(SourcePath_.path))));
TypedQuery<SourcePath> query = entityManager.createQuery(pathQuery);
query.getResultList();
The resulting sql query results something like:
select ...snip aliases... from SourcePath where cast(x'00010002', char(1)) = path;
(path would be some nasty old alias, but that's irrelevant).
This query is incorrect. Particularly, the cast: cast(x'00010002', char(1)) is not a cast to as string, as specified by .as(String.class), instead it should be either cast(x'00010002', char), or cast(x'00010002', char(N) where N is a suitably big enough number.
I've isolated the cause of this cast failure to the MySqlDialect provided by org.hibernate. Particularly:
public String getCastTypeName(int code) {
if ( code==Types.INTEGER ) {
return "signed";
}
else if ( code==Types.VARCHAR ) {
return "char";
}
...snip...
}
Which farther down the chain is interpreted as a char, which is registered by the dialog: registerColumnType( Types.CHAR, "char(1)" );.
Finally, to my question. How can I work around this behaviour? Do I report it as a bug to Hibernate? Do I extend the Dialog and correct the returned type from getCastTypeName? Is there an alternative to .as which will appropriately cast? Or, should I use strings everywhere I'm currently using hex-values to avoid touching edge cases of hibernate's implementation?
Thanks
idb
IMHO, you should use a String literal like cb.literal("\u0001\u0002"). Reasons:
It's the same thing as cb.literal(0x00010002).as(String.class), but less verbose.
Does not hit any "edge case"
It's clearer: does "0x00010002" be treated in Big Endian or LE? Which encoding should be used?
You can improve legibility using constants (e.g.: cb.literal(ASCII_SOH + ASCII_STX) where SOH="\u0001" and STX="\u0002").
EDIT: Adding a better description, since I didn't see the "Or, should I use strings everywhere I'm currently using hex-values"

MySQL - Perl: How to use an array with IN within a select query? (WHERE IN (#array))

This is an addition to my solved question here:
how to get array of zip codes within x miles in perl
OK, I have the array #zips. Now I am trying to use it in a query like this:
SELECT `club_name`,`city` FROM `table` WHERE `public_gig` = 'y' AND `zip` IN (#zips)
#I also tried syntax "IN ("#zips"), IN #zips and IN ('#zips')"
But, I cannot get it to work. (I am using placeholders and such as you see in my link above.)
I was able to get this to work:
$fzip=shift(#Zips);
$lzip=pop(#Zips);
SELECT `club_name`,`city` FROM `table` WHERE `public_gig` = 'y' AND `zip` BETWEEN $fzip AND $lzip
ZIP | public_gig | start_time | fin_time | city | club_name | and so on
33416 | y | 9pm | 2am | clearwater | beach bar | yada
But, for obvious reasons and some resemblance of accuracy, that is not really what I want. Just wanted to see if I could get SOMETHING working on my own.
Why can't I get the query to work with the zips in the array using IN?? Nothing is returned and there is no error.
There is actually a lot more in that query but, I left it all out to keep it short here.
I tried to figure it out by myself. Obviously, my learning capacity for the day is near peak.
Thanks for any help.
All of the examples posted here will screw up if any of your values contain single-quotes, don't use them.
Instead (assuming $dbh is the database handle for your mysql connection):
my $zip_string = join q{,}, map $dbh->quote($_), #zips;
and interpolate that.
Or, for something nice, but not half as outlandish as DBIx::Perlish: SQL::Abstract.
my $sqla = SQL::Abstract->new;
my ($sql, #bind) = $sqla->select(
'table',
['club_name', 'city'],
{
public_gig => y',
zip => { -in => \#zips },
}
);
$dbh->prepare($sql);
$dbh->execute(#bind);
# fetchrow etc.
This can be done using placeholders, you just have to work around the limitation that each placeholder can only accept a single value. WHERE zip IN (?) won't work because you're (presumably) looking for more than one value (otherwise, why use IN?).
You can, however, easily build a statement on the fly with the correct number of placeholders:
#!/usr/bin/env perl
use strict;
use warnings;
my #zips = (12345, 54321, 90210);
my $stmt = "SELECT `club_name`,`city`
FROM `table`
WHERE `public_gig` = 'y' AND `zip` IN ("
. join(', ', ('?') x #zips) . ')';
print "$stmt\n";
# Now just:
# my $sth = $dbh->prepare($stmt);
# $sth->execute(#zips);
Alternatively, if you don't mind using weird CPAN modules,
with DBIx::Perlish you can just say:
my #results = db_fetch {
my $t: table;
$t->public_gig eq "y";
$t->zip <- #zips;
};
and it will do the right thing.
Full disclosure: I am the author of DBIx::Perlish.
I don't know perl too much, but this looks like a simple SQL problem: why don't you just build the SQL IN clause from your array? You should get something like
AND zip IN ('zip 1', 'zip 2', '...')
I doubt just adding an array in perl will create the right strings for the SQL string ...
You need to turn the array into a string of values seperated by commas.
Try this :
my $zipcodes = join('\',\'',#zips);
SELECT `club_name`,`city` FROM `table` WHERE `public_gig` = 'y' AND `zip` IN ('".$zipcodes."');