Within a table, there is a params row that has some JSON encoded data -- as such:
{"categories":211,"singleCatOrdering":"","menu-anchor_title":"","menu-anchor_css":"","menu_image":"","menu_text":1,"page_title":"","show_page_heading":0,"page_heading":"","pageclass_sfx":"","menu-meta_description":"","menu-meta_keywords":"","robots":"","secure":0}
What I need to do is wrap [] around the categories parameter - to look as such:
{"categories":[211],"singleCatOrdering":"","menu-anchor_title":"","menu-anchor_css":"","menu_image":"","menu_text":1,"page_title":"","show_page_heading":0,"page_heading":"","pageclass_sfx":"","menu-meta_description":"","menu-meta_keywords":"","robots":"","secure":0}
I have tried the following (and a bunch of other failed ones) with no-dice:
UPDATE j17_menu SET params = REPLACE(params,'"categories":%,','"categories":[%],') WHERE component_id = 10021;
Am I possibly using the wildcard option wrong? Any nudges in the right direction would be a huge help. Thanks!
This one will work independent of the context
UPDATE j17_menu
SET params=CONCAT (
SUBSTR(params,1,LOCATE('"categories":',params)),
'"categories":[',
substr(
params,
LOCATE('"categories":',params)+13,
LOCATE(',',params,LOCATE('"categories":',params))-LOCATE('"categories":',params)-13
),
']',
substr(params,LOCATE(',',params,LOCATE('"categories":',params)))
)
WHERE component_id = 10021;
Related
I'm trying to read a .csv file with the following format using MAC:
;lon;lat
0;55,245594;25,066697
1;55,135613;25,070419
2;55,275683;25,203425
What I am doing so far is:
$call csv2gdx coords.csv id=d index=1 values=2..lastCol useHeader=y
sets
i
c /x,y/
;
parameters
dloc(i,c) 'locations'
;
$gdxin clients_csv.gdx
$load ___ ?
What I want to do is read the lat,lon coordinates in the parameter dloc so as for each i to have a pair of coords c, i.e. lat, lon.
Example output:
x y
i1 17.175 84.327
Running your code produces an error from csv2gdx:
*** ErrNr = 15 Msg = Values(s) column number exceeds column count; Index = 2, ColCnt = 1
Per default, csv2gdx expects the entries separated by commas, which you do not have in your data. You could also define semicolon or tab as separator by means of an option, but if the data has really the format you posted, you do not need to call csv2gdx at all. You could just include the data directly like this:
Sets
i
c
;
Table dloc(i<,c<) 'locations'
$include coords.csv
;
Display dloc;
EDIT after change of input data format:
The error message is still the same. And also the reason is the same: You use a different field separator than the default one. If you switch that using the option fieldSep=semiColon, you will realize that also your decimal separator is non-default for csv2gdx. But this can be changed as well. Here is the whole code (with adjusted csv2gdx call and adjustments for data loading). Note that sets i and c get implicitly defined when loading dloc with the < syntax in the declaration of dloc.
$call csv2gdx coords.csv id=d index=1 values=2..lastCol useHeader=y fieldSep=semiColon decimalSep=comma
Sets
i
c
;
parameters
dloc(i<,c<) 'locations'
;
$gdxin coords.gdx
$load dloc=d
Display dloc;
$exit\
Hi here is a small snipit of a very large perl program I am working on, however I believe the error is around this part.
my $headd = "test,test1,test2,test3,ect";
my #headers = split(',', $headd);
CONNECT TO DATABASE STUFF THAT WORKS FINE
my %column_mapping = (
"GPS ALT" => 'GPS_ALT',
"GPS DTS" => 'GPS_DTS',
"GPS FIX" => 'GPS_FIX',
"GPS HDG" => 'GPS_HDG'
)
my $sql = sprintf 'INSERT INTO tablename ( %s ) VALUES ( %s )',
join( ',', map { $column_mapping{$_} } #$headers ),
join( ',', ('?') x scalar #$headers ); # note the list around the '?'
Since I am fairly new to perl, if I had to guess I would say its around here
join( ',', map { $column_mapping{$_} } #$headers ),
join( ',', ('?') x scalar #$headers );
I think I have the wrong data types for headers or something, but I am not entirely sure what the problem is, it just crashes around those lines.
If you happen to see something I did wrong that would be great:)
Your main error is in not adding use strict and use warnings to the top of your Perl code. These lines will find a number of problems that often sneak into Perl programs. They would have found this problem.
The problem is that you declare an array called #headers.
my #headers = split(',', $headd);
But the next time you try to use it, you are assuming that it is an array reference called $headers.
join( ',', map { $column_mapping{$_} } #$headers )
In Perl, $headers and #headers are two completely different variables with absolutely no connection between them. As you have never given $headers a value, trying to dereference it (#$headers) was never going to go well.
But you've worked that out from the comments. You've made the suggested fix (replace #$headers with #headers) and now you get a different error:
the problem is saying I have an initialized value in the join
I assume you mean "uninitialized" :-)
Sounds like the problem is on the same line that we discussed above (now with the fixed syntax):
join( ',', map { $column_mapping{$_} } #headers )
So what's happening here? Well, we are effectively translating values to new values. For each value in #headers we look it up in %column_mappings and return the associated value from the hash.
How could that give us an undefined value? Well, what happens if we look up a key in a hash that doesn't actually exist in the hash? Perl gives us the special value "undef". Which would trigger the warning that you are getting.
So my suggestion to you is that you double-check the values that you are getting in #headers (which will be the values from $headd) and make sure that all of those possible values have an associated key in %column_mappings.
One workaround would be to set a default value in the map. Something like this perhaps:
join( ',', map { $column_mapping{$_} // 'MISSING MAPPING' } #$headers )
That will almost certainly break your database interaction at some point further down the line - but at least it will be obvious what the problem is!
Update: To eliminate the problematic values before they get into #headers:
my #headers = grep { exists $column_mapping{$_} } split(',', $headd);
You'll want to move this statement after the definition of %column_mappings (for hopefully obvious reasons).
My table has a column with a JSON string that has nested objects (so a simple REPLACE function cannot solve this problem) . For example like this: {'name':'bob', 'blob': {'foo':'bar'}, 'age': 12}. What is the easiest query to append a value to the end of the JSON string? So for the example, I want the end result to look like this: {'name':'bob', 'blob': {'foo':'bar'}, 'age': 12, 'gender': 'male'} The solution should be generic enough to work for any JSON values.
What about this
UPDATE table SET table_field1 = CONCAT(table_field1,' This will be added.');
EDIT:
I personally would have done the manipulation with a language like PHP before inserting it. Much easier. Anyway, Ok is this what you want? This should work providing your json format that is being added is in the format {'key':'value'}
UPDATE table
SET col = CONCAT_WS(",", SUBSTRING(col, 1, CHAR_LENGTH(col) - 1),SUBSTRING('newjson', 2));
I think you can use REPLACE function to achieve this
UPDATE table
SET column = REPLACE(column, '{\'name\':\'bob\', \'blob\': {\'foo\':\'bar\'}, \'age\': 12}', '{\'name\':\'bob\', \'blob\': {\'foo\':\'bar\'}, \'age\': 12, \'gender\': \'male\'}')
Take care to properly escape all quotes inside json
Upon you request of nested json, i think you can just remove last character of the string with SUBSTRING function and then append whatever you need with CONCAT
UPDATE table
SET column = CONCAT(SUBSTRING(column, 0, -1), 'newjsontoappend')
modify Jack's answer. Works perfectly even column value is empty on first update.
update table
set column_name = case when column_name is null or column_name =''
then "{'foo':'bar'}"
else CONCAT_WS(",", SUBSTRING(column_name, 1, CHAR_LENGTH(column_name) - 1),SUBSTRING("{'foo':'bar'}", 2))
end
Very similar to this question MySQL Dynamic Query Statement in Python
However what I am looking to do instead of two lists is to use a dictionary
Let's say i have this dictionary
instance_insert = {
# sql column variable value
'instance_id' : 'instnace.id',
'customer_id' : 'customer.id',
'os' : 'instance.platform',
}
And I want to populate a mysql database with an insert statement using sql column as the sql column name and the variable name as the variable that will hold the value that is to be inserted into the mysql table.
Kind of lost because I don't understand exactly what this statement does, but was pulled from the question that I posted where he was using two lists to do what he wanted.
sql = "INSERT INTO instance_info_test VALUES (%s);" % ', '.join('?' for _ in instance_insert)
cur.execute (sql, instance_insert)
Also I would like it to be dynamic in the sense that I can add/remove columns to the dictionary
Before you post, you might want to try searching for something more specific to your question. For instance, when I Googled "python mysqldb insert dictionary", I found a good answer on the first page, at http://mail.python.org/pipermail/tutor/2010-December/080701.html. Relevant part:
Here's what I came up with when I tried to make a generalized version
of the above:
def add_row(cursor, tablename, rowdict):
# XXX tablename not sanitized
# XXX test for allowed keys is case-sensitive
# filter out keys that are not column names
cursor.execute("describe %s" % tablename)
allowed_keys = set(row[0] for row in cursor.fetchall())
keys = allowed_keys.intersection(rowdict)
if len(rowdict) > len(keys):
unknown_keys = set(rowdict) - allowed_keys
print >> sys.stderr, "skipping keys:", ", ".join(unknown_keys)
columns = ", ".join(keys)
values_template = ", ".join(["%s"] * len(keys))
sql = "insert into %s (%s) values (%s)" % (
tablename, columns, values_template)
values = tuple(rowdict[key] for key in keys)
cursor.execute(sql, values)
filename = ...
tablename = ...
db = MySQLdb.connect(...)
cursor = db.cursor()
with open(filename) as instream:
row = json.load(instream)
add_row(cursor, tablename, row)
Peter
If you know your inputs will always be valid (table name is valid, columns are present in the table), and you're not importing from a JSON file as the example is, you can simplify this function. But it'll accomplish what you want to accomplish. While it may initially seem like DictCursor would be helpful, it looks like DictCursor is useful for returning a dictionary of values, but it can't execute from a dict.
I have the following query:
select subclasses.id,participants_subclasses.participant_id
from subclasses
left outer join participants_subclasses on
participants_subclasses.participant_id = ?
and subclasses.id = participants_subclasses.subclass_id
where
subclasses.classification_id = ?
and subclasses.showhover
order by subclasses.seq,
IF(LEFT(subclasses.code, 1) = '<',
Extractvalue(subclasses.code, "//texts/text/content"),
subclasses.code)
The above query is processing a table where the code column sometimes has text and sometimes has XML with text inside a tag. The above query works. The side-effect is that a code value cannot start with a "<" which should be acceptable, but the order by would mistake it for XML content. The query below would be more specific and accurate:
select subclasses.id,participants_subclasses.participant_id
from subclasses
left outer join participants_subclasses on
participants_subclasses.participant_id = ?
and subclasses.id = participants_subclasses.subclass_id
where
subclasses.classification_id = ?
and subclasses.showhover
order by subclasses.seq,
IF(LEFT(subclasses.code, 5) = '<?xml',
Extractvalue(subclasses.code, "//texts/text/content"),
subclasses.code)
However this variation checking the XML header in the content fails with a "NameInput Array does not match ?" error in MySQL. It appears that the ? inside <?xml literal is being mistaken for a bind target. And I am passing 2 values to be bound - which again is correct.
So my question is - how do I get the <?xml literal to not be mistaken for a bind value target???
SOLVED
This turns out to be a bug in ADODB interface from phpLens, and NOT in MySQL itself. It exists in current version which is 5.17 for PHP5.