I am using a MySql stored procedure, get_list, to populate a ttk.Combobox
my_number = tk.StringVar()
cb=ttk.Combobox(top_frame,width=25,textvariable=my_number)
cb['values']=db.get_list()
cb.grid(row=1,column=4)
It works fine, except the results are bracketed by {}.
RESULTS: {Smith, John}
What I want: Smith, John
How do I get rid of the beginning and ending {}?
The problem is that the database query is returning a list of rows, where each row is a list with a single column. What you need for the values option is a list of strings.
So, the simple solution is to use a list comprehension to convert each row to a string. Something like this should work:
values = [row[0] for row in db.get_list()]
cb['values'] = values
Since I can't run your code I can't say for sure. Regardless, the root of the problem is that you're passing a list of lists (or tuples) and tkinter requires a list of strings.
Because internally tkinter uses curly braces for lists, it's adding the curly braces to preserve the nature of the data you're passing in. It's up to you to convert the data you get back from db.get_list() into a list of strings.
Bryan, I inserted the two line of code you suggested, but I am still getting value surrounded by {}. Lots I don't understand, but I believe, never know for sure, that the MySql procedure is returning a string. Here is the MySQL code:
CREATE DEFINER=`root`#`localhost` PROCEDURE `get_list`()
BEGIN
SELECT concat(last_name,', ', first_name) as last_first
FROM volunteers
ORDER BY
last_first;
END
my_number = tk.StringVar()
cb = ttk.Combobox(top_frame, width=25, textvariable=my_number)
values = [row[0] for row in db.get_list()]
cb['values'] = db.get_list()
cb.grid(row=1, column=4)
thanks for your persistence.
Related
I am fetching data from a third party API, which responds with a JSON payload. However, this JSON contains another JSON object, stored as a string including escape characters. Example:
{
"aggregationType": "IDENTITY",
"outputs": [
{
"name": "Sinusoid|Sinusoid"
}
],
"value": "{\"dataX\":[1,2,3,4],\"dataY\":[1,4,9,16]}"
}
In the first part of the file, we have some regular parameters like 'aggregationType' and 'outputs', but the last parameter 'value' is the JSON object I am talking about.
What I would like to do is to enter the 'dataX' and 'dataY' arrays together into a table on a SQL DB. I haven't found a straightforward way of doing it so far.
What I've tried:
Using a simple copy activity, but I can only access the whole 'value' field, not separate out 'dataX' from 'dataY', let alone the array's individual values.
Using the lookup activity to then store 'value' in a variable. From here I can get to a usable JSON object in ADF, but the only way I've found of then sending the data to the DB is to use a ForEach activity containing a copy activity. Since dataX and dataY are in reality much larger, this seems to take forever when I debug.
Copying only the 'value' object to a blob and trying to retrieve the data from there. This hasn't worked because the object always ends up getting stored with the initial " marks and the \ escape characters.
Is there any way of getting around this issue?
You can store the value in a staging kind of table in SQL and then create a stored procedure to separate out the objects as arrays
JSON_Value can help you extract values:
SELECT JSON_VALUE('{"dataX": [1,2,3,4]}', '$.dataX') AS 'Output';
In your stored procedure you can try using above query and insert values in SQL table
To expand, on the tip from #Pratik Somaiya, I've written up a stored procedure that does the work of inserting the data to a persistent table.
I've had to use a WHILE loop, which doesn't really feel right, so I'm still on the lookout for a better solution on that.
CREATE OR ALTER PROCEDURE dataset_outputs_from_adf
#json_output_value NVARCHAR(MAX),
#output_name NVARCHAR(100)
AS
DECLARE #i INT = 0
WHILE JSON_VALUE(#json_output_value,CONCAT('$.dataX[',#i,']')) IS NOT NULL
BEGIN
INSERT INTO my_table
VALUES (
#output_name,
JSON_VALUE(
#json_output_value,
CONCAT('$.dataX[',#i,']')
),
JSON_VALUE(
#json_output_value,
CONCAT('$.dataY[',#i,']')
)
)
SET #i = #i + 1
END
GO
I should be able to make the whole thing repeatable without repetitive code by parameterizing Data Factory with the output name.
As I understand it, you can retrieve the embedded "value" JSON but it retains its escape characters. You would like to pass the arrays [1,2,3,4] and [1,4,9,16] to a relational database (Microsoft SQL Server?) to store them.
The embedded "value" can be converted to referencable JSON using the expression json(). This handles the escaped characters.
#json(variables('payload')).dataX
Will return the array [1,2,3,4] as expected.
How best to get this into SQL Server? We are limited to the activities ADF supports, which really comes down to a stored procedure (SP). Using a table valued parameter would be ideal, but not possible in current ADF. So I would suggest passing it to the SP as a string.
#string(json(variables('payload')).dataX)
This will look much the same as above but will be a string not an array.
In the SP there are a couple of ways to parse this string. If your version supports it STRING_SPLIT is convenient. Note the passed string will retain its leading and trailing square bracket. These can be removed in ADF or in SQL, it doesn't much matter where.
Since the data is JSON it may make more sense to use OPENJSON instead. Let's say we pass the contents of dataX to a SP parameter #dataX varchar(4000). Inside the SP we write
create procedure dbo.HandleData
#dataX varchar(4000),
#dataY varchar(4000)
as
insert dbo.SomeTable(ColumnX, ColumnY)
select x.value, y.value
from OPENJSON(#dataX) x
inner join OPENJSON(#dataY) y
on y.[key] = x.[key];
Further code may be needed if the arrays could be of different lengths, or there are NULLs, or non-integer values etc. Of course the resultset from OPENJSON can be used for joining or any other purpose within the SP.
If both dataX and dataY from the original payload end up in the same DB table the SP can be called twice, once for each. Alternatively you can union their arrays in ADF and call the SP once.
I have one column(varchar) containing only json string within one table. I want replace all keys with "" on that column. How can I do that using sql? My database is MySQL.
For example:
|--------------------------------------------------------------------|
| t_column |
|--------------------------------------------------------------------|
| {"name":"mike","email":"xxx#example.com","isManage":false,"age":22}|
|--------------------------------------------------------------------|
SELECT replace(t_column, regexp, "") FROM t_table
I expect:
mikexxx#example.comfalse22
How to write that regexp?
Start from
select t_column->'$.*' from test
This will return a JSON array of attribute values:
[22, "mike", "xxx#example.com", false]
This might be already all you need, and you can try something like
select *
from test
where t_column->'$.*' like '%mike%';
Unfortunately there seems to be no native way to join array values to a single string like JSON_ARRAY_CONCAT(). In MySQL 8.0 you can try REGEXP_REPLACE() and strip all JSON characters:
select regexp_replace(t_column->'$.*', '[" ,\\[\\]]', '') from test
which will return '22mikexxx#example.comfalse'.
If the values can contain one of those characters, they will also be removed.
Note: That isn't very reliable. But it's all I can do in a "simple" way.
See demo on db-fiddle.
I could be making it too simplistic, but this is just a mockup based on your comment. I can formalize it into a query if it fits your requirement.
Let's say you get your JSON string to this format where you replace all the double quotes and curly brackets and then add a comma at the end. After playing with replace and concat_ws, you are now left with:
name:mike,email:xxx#example.com,isManage:false,age:22,
With this format, every value is now preceded by a semicolon and followed by a comma, which is not true for the key. Let's say you now want to see if this JSON string has the value "mike" in it. This, you could achieve using
select * from your_table where json_col like '%:mike,%';
If you really want to solve the problem with your approach then the question becomes
What is the regex that selects all the undesired text from the string {"name":"mike","email":"xxx#example.com","isManage":false,"age":22} ?
Then the answer would be: {\"name\":\"|\"email\":\"|\",\"isManage\":|,\"age\":|}
But as others let you notice I would actually approach the problem parsing JSONs. Look up for functions json_value and json_query
Hope I helped
PS: Keep close attention on how I structured the bolded sentence. Any difference changes the problem.
EDIT:
If you want a more generic expression, something like select all the text that is not a value on a json-formatted string, you can use this one:
{|",|"\w+\":|"|,|}
I have an JSON string stored in the database and I need to SQL COUNT based on the WHERE condition that is in the JSON string. I need it to work on the MYSQL 5.5.
The only solution that I found and could work is to use the REGEXP function in the SQL query.
Here is my JSON string stored in the custom_data column:
{"language_display":["1","2","3"],"quantity":1500,"meta_display:":["1","2","3"]}
https://regex101.com/r/G8gfzj/1
I now need to create a SQL sentence:
SELECT COUNT(..) WHERE custom_data REGEXP '[HELP_HERE]'
The condition that I look for is that the language_display has to be either 1, 2 or 3... or whatever value I will define when I create the SQL sentence.
So far I came here with the REGEX expression, but it does not work:
(?:\"language_display\":\[(?:"1")\])
Where 1 is replaced with the value that I look for. I could in general look also for "1" (with quotes), but it will also be found in the meta_display array, that will have different values.
I am not good with REGEX! Any suggestions?
I used the following regex to get matches on your test string
\"language_display\":\[(:?\"[0-9]\"\,)*?\"3\"(:?\,\"[0-9]\")*?\]
https://regex101.com/ is a free online regex tester, it seems to work great. Start small and work big.
Sorry it doesn't work for you. It must be failing on the non greedy '*?' perhaps try without the '?'
Have a look at how to serialize this data, with an eye to serializing the language display fields.
How to store a list in a column of a database table
Even if you were to get your idea working it will be slow as fvck. Better off to process through each row once and generate something more easily searched via sql. Even a field containing the comma separated list would be better.
I have tried using 'LIKE' but it runs into problems which i will explain below.
i have a string column that looks like any of these.
"1010, 2020, 3030"
"1010"
""
I want to be able to see if this string contains a single ID. e.g 2020. if it does then return the row. I tried using like but if the id is 20 it will return the row because 2020 contains 20.
Selecting the entire db and then using a delimiter to go through all the strings will take far too much time. Is it possible to implement this?
This is why you don't store multiple values in a single field. Because your bad design, this is the query structure you'll have to use EVERY SINGLE TIME to compensate for it:
WHERE
foo = 2020 // exact match, only value in field
OR foo LIKE '2020,%' // value is at start of field
OR foo LIKE '%,2020,%' // value is somewhere in the middle of the field
OR foo LIKE '%,2020' // value is at the end of the field
Or you could have had a properly normalized design, and just done
WHERE childtable.foo = 2020
and be done with it.
First, you should not store lists of things in string variables. SQL has a very nice data structure for lists. It is called a table. Each row in such a table would have an id and one value from the list.
That said, sometimes you are stuck with data like this. In that case, you can use find_in-set():
where find_in_set('20', replace(stringcolumn, ', ', ',')) > 0;
You can also do the logic with like, but MySQL has the convenient built-in function for this.
EDIT:
If you want to do this with like:
where concat(',', stringcolumn, ',') like '%,20,%'
Note that the delimiters "protect" the values, so 20 is not confused with 2020.
I have two databases, both containing phone numbers. I need to find all instances of duplicate phone numbers, but the formats of database 1 vary wildly from the format of database 2.
I'd like to strip out all non-digit characters and just compare the two 10-digit strings to determine if it's a duplicate, something like:
SELECT b.phone as barPhone, sp.phone as SPPhone FROM bars b JOIN single_platform_bars sp ON sp.phone.REGEX = b.phone.REGEX
Is such a thing even possible in a mysql query? If so, how do I go about accomplishing this?
EDIT: Looks like it is, in fact, a thing you can do! Hooray! The following query returned exactly what I needed:
SELECT b.phone, b.id, sp.phone, sp.id
FROM bars b JOIN single_platform_bars sp ON REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(b.phone,' ',''),'-',''),'(',''),')',''),'.','') = REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(sp.phone,' ',''),'-',''),'(',''),')',''),'.','')
MySQL doesn't support returning the "match" of a regular expression. The MySQL REGEXP function returns a 1 or 0, depending on whether an expression matched a regular expression test or not.
You can use the REPLACE function to replace a specific character, and you can nest those. But it would be unwieldy for all "non-digit" characters. If you want to remove spaces, dashes, open and close parens e.g.
REPLACE(REPLACE(REPLACE(REPLACE(sp.phone,' ',''),'-',''),'(',''),')','')
One approach is to create user defined function to return just the digits from a string. But if you don't want to create a user defined function...
This can be done in native MySQL. This approach is a bit unwieldy, but it is workable for strings of "reasonable" length.
SELECT CONCAT(IF(SUBSTR(sp.phone,1,1) REGEXP '^[0-9]$',SUBSTR(sp.phone,1,1),'')
,IF(SUBSTR(sp.phone,2,1) REGEXP '^[0-9]$',SUBSTR(sp.phone,2,1),'')
,IF(SUBSTR(sp.phone,3,1) REGEXP '^[0-9]$',SUBSTR(sp.phone,3,1),'')
,IF(SUBSTR(sp.phone,4,1) REGEXP '^[0-9]$',SUBSTR(sp.phone,4,1),'')
,IF(SUBSTR(sp.phone,5,1) REGEXP '^[0-9]$',SUBSTR(sp.phone,5,1),'')
) AS phone_digits
FROM sp
To unpack that a bit... we extract a single character from the first position in the string, check if it's a digit, if it is a digit, we return the character, otherwise we return an empty string. We repeat this for the second, third, etc. characters in the string. We concatenate all of the returned characters and empty strings back into a single string.
Obviously, the expression above is checking only the first five characters of the string, you would need to extend this, basically adding a line for each position you want to check...
And unwieldy expressions like this can be included in a predicate (in a WHERE clause). (I've just shown it in the SELECT list for convenience.)
MySQL doesn't support such string operations natively. You will either need to use a UDF like this, or else create a stored function that iterates over a string parameter concatenating to its return value every digit that it encounters.