I want to convert the rows of a record set to JSON, but not include any null entries that are just going to end up being undefined in JavaScript anyway. For example, suppose I have the table testdata with entries
id | prop1 (integer) | prop2 (text)
-------------------------------------
1 | 42 | 'Answer'
2 | NULL | 'No prop one'
3 | 0 | NULL
and then execute
SELECT row_to_json(testdata) FROM testdata
What I get is:
{"id":"1","prop1":"42","prop2":"Answer"}
{"id":"2","prop1":null,"prop2":"No prop one"}
{"id":"3","prop1":"0","prop2":null}
But instead, what I want is:
{"id":"1","prop1":"42","prop2":"Answer"}
{"id":"2","prop2":"No prop one"}
{"id":"3","prop1":"0"}
Is this possible? According to the JSON functions documentation for PostgreSQL 9.3, there's only one extra option or parameter for row_to_json, but setting pretty_bool=true doesn't remove the nulls, so it seems as if the answer may be no. But this also seems as if it's a very obvious and useful function, so I'm hoping that somebody else has found something I've missed.
My end goal is to retrieve the records in JavaScript with a GET call to a PHP page. Am I better off building the JSON in PHP from a more standard recordset, instead of using PostgreSQL's JSON routines?
Postgres 9.5 introduces json_strip_nulls function, that seems to do exactly what you want.
Looks like future versions of the function may have an 'ignore nulls' option - https://commitfest.postgresql.org/action/patch_view?id=1496
Related
I have a table which's name is users in my MySQL database, and I am using this DB with Ruby on Rails application with ORM structure for years. The table has id field and this field is configured as AI (auto-increment), BIGINT.
Example of my users table;
+----+---------+
| id | name |
+----+---------+
| 1 | John |
| 2 | Tommy |
| 3 | ... |
| 4 | ... |
| 5 | ... |
| 6 | ... |
+----+---------+
The problem I am facing is when I execute the following query I get unexpected rows.
SELECT * FROM users WHERE id = '1AW3F4SEFR';
This query is returning the exact same value with the following query,
SELECT * FROM users WHERE id = 1;
I do not know why SQL let me use strings in WHERE clause on a data type INT. And as we can see from the example, my DB converts the strings I gave to the integer at position 0. I mean, I search for 1AW3F4SEFR and I expect not to get any result. But SQL statement returns the results for id = 1.
In Oracle SQL, the behavior of this exact same query is completely different. So, I believe there is something different on MySQL. But I am not sure about what causes this.
As has been explained in the request comments, MySQL has a weird way of converting strings to numbers. It simply takes as much of a string from the left as is numeric and ignores the rest. If the string doesn't start with a number the conversion defaults to 0.
Examples: '123' => 123, '12.3' => 12.3, '.123' => 0.123, '12A3' => 12, 'A123' => 0, '.1A1.' => 0.1
Demo: https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=55cd18865fad4738d03bf28082217ca8
That MySQL doesn't raise an error here as other DBMS do, can easily lead to undesired query results that get a long time undetected.
The solution is easy though: Don't let this happen. Don't compare a numeric column with a string. If the ID '1AW3F4SEFR' is entered in some app, raise an error in the app or even prevent this value from being entered. When running the SQL query, make sure to pass a numeric value, so '1AW3F4SEFR' cannot even make it into the DBMS. (Look up how to use prepared statements and pass parameters of different types to the database system in your programming language.)
If for some reason you want to pass a string for the ID instead (I cannot think of any such reason though) and want to make your query fail-safe by not returning any row in case of an ID like '1AW3F4SEFR', check whether the ID string represents an integer value in the query. You can use REGEXP for this.
SELECT * FROM users WHERE id = #id AND #id REGEXP '^[0-9]+$';
Thus you only consider integer ID strings and still enable the DBMS to use an index when looking up the ID.
Demo: https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=56f8ee902342752933c20b8762f14dbb
I have the following table:
+-------------+----------------+
| id | server |
+-------------+----------------+
| 1 | ["1", "15"] |
+-------------+----------------+
I need to get only value that is grather that 1 so in above example i need to get from output only 15
I try using this:
SELECT
JSON_EXTRACT(server, "$[*]") as server
FROM streams
WHERE JSON_EXTRACT(server, "$[*]") != JSON_QUOTE('1')
AND id=1;
But i always get ["1", "15"] and need to get ["15"].
Unfortunately, you can't do this with MySQL's JSON_EXTRACT and JSON_SEARCH functions as they perform extraction and exact matching (not comparison) respectively. So, you have two options:
Normalise the table and have server values into a new column (recommended)
Fetch all the values and perform the filtering in service layer
I added this:
SELECT CASE WHEN
JSON_UNQUOTE(JSON_SEARCH(server, 'all', 1)) IS NULL THEN
JSON_REMOVE(server, '$."1"')
ELSE
JSON_REMOVE(server, JSON_UNQUOTE(JSON_SEARCH(server, 'one', 1)))
END AS server
FROM streams WHERE id=2 AND server NOT LIKE '%[]%';
Now it works but if ["1"] is only 1 in json column i get []...a added NOT LIKE '%[]%' but it prints out always []...where i need to add to get no results found from mysql?
Best way to store a dynamic expression in a table for each row for a searching module.
The expression is dynamic and can have multiple fields which are being compared.
I considered creating a separate column for each type of field and fattening out complex nested logic by getting all possible combinations using dnf and storing them in my table. The disadvantages of doing that is for every new logic and expression, a new column has to be created which would lead to a large table which has too many NULLS in it and also adding a new column would take time & refactoring(we are talking about more than 800 columns here).
The alternate approach which I think would work better is below->
I want to discuss if there are better way to this, and if not, how can we improve and achieve the below suggested approach.
| id | expression | diagnosis |
|------|------------------------------------------------|-------------|
| 1 |`p.age>12 and p.gender==Male` | diseaseA |
| 2 |`p.age>50 and p.bp>20` | diseaseB |
| 3 |`p.age<20 and p.bp<20` | diseaseC |
| 4 |`p.age<30 and p.age>20 and (p.bp<30 or p.bp>50)`| diseaseD |
I want to search in this table, for a patient p with certain properties (age=*something*,bp=*something*,etc).
The resulting rows should return all rows which satisfy the expression and also rows which partially match the expression(i.e the rows which are using properties not supplied in the search criteria).
For example for a search for patient p(age=22,bp=15), the search result should be
| id | disease |
|------|-------------|
| 1 | diseaseA |
| 3 | diseaseC |
| 4 | diseaseD |
Since I am new to SQL, the (newbie) way I think I can do this is
First get all the rows(in-memory would be costly, lets discuss what is best possible way to execute the below said functionality in point 2 row-by-row)
Then row-by-row transform the expression to a logical executable expression(which is later executed using eval) using regex matching & replacement(I hope there is a better way than this) for the search criteria(i.e. substituting the patient details) [in my example for the 2nd row, the expression p.age>50 and p.bp>20 gets converted to "22>50 && 15>20"]
All the rows for which the result of transforming & executing the result was true(or partially matched) should be returned.
The language is not an issue as I would be starting this project from scratch and can use any language
I can answer for MySQL.
First of all, you'll have to write all of your sql code inside sql procedure.
Generally you are interestedin dynamic SQL
https://dev.mysql.com/doc/refman/5.7/en/sql-syntax-prepared-statements.html
So a straight-forward approach is to open a a cursor for your table with expressions and for each expression replace p.age with it's actual value and then execute dynamic SQL. (select 22 > 50 and 15 > 20)
Another approach is to loop through expression table (open cursor for it) and as you probably have patient id (not only it's field values) just generate normal sql that selects from patient table (select patient_id from patients where [expression_from_expression_table] and patient_id = [your_known_patient_id])
And the third one that I can imagine is generating a big single query from whole expression table
select group_concat(concat('if(', expression, ',"', diagnosis, '", "") as ', diagnosis) separator ',') from expressions into somevar;
and then doing replace of p.* with actual values and executing second query:
set somevar = replace(somevar, 'p.age', '15');
...
#qry = concat('select ', somevar);
PREPARE qry FROM #qry;
EXECUTE qry;
The third approach is fastest to my mind but will require aditional work on client as you will recieve diagnosis as columns, not as rows.
But hope you get the general idea.
In Access db I have a linked table that references Excel file.
In the Excel file I have 2 columns:
Col1 | Col2
---------------
date1 | =if(Col1="","",Col1+1) -> Evaluates to date1+1
<blank> | =if(Col1="","",Col1+1) -> Evaluates to ""
In Access I see it as
Col1 | Col2
---------------
date1 | date1+1
<null> | #Num!
I can't find a way to deal with the problem. The idea is to end up having <null> instead of the error value. Can I capture this error in Access? I have tried looking for error capturing function but I found nothing. I can think of workaround like returning 0 instead of "" and then filtering it out in Access but it doesn't seem like a proper way of doing it.
I could also use the first column to filter the second but again it doesn't seem proper, because in some other cases I could have just 1 column.
IIf evaluates both expressions, and don't mix dates and strings, so try this:
=IIf(IsNull(Col1),Null,DateAdd("d",1,Nz(Col1, Date()))
or:
=CVDate(Col1)+1
I'm having a problem with a column ( VARCHAR(513) NOT NULL ) on a MySQL table.During a procedure of import from a CSV file, a bunch of rows got filled with some weird stuff coming from I don't know where.
This stuff is not visible from Workbench, but if I query the DBMS with:SELECT * FROM MyTable;I got:ID | Drive | Directory | URI | Type ||
1 | Z: | \Users\Data\ | \server\dati | 1 || // <-correct row
...
32 | NULL | \Users\OtherDir\ | | 0 ||While row 1 is correct, row 32 shows a URI filled with something. Now, if I query dbms with:SELECT length(URI) FROM MyTable WHERE ID = 32; I got 32. While, doing:SELECT URI FROM MyTable WhERE ID = 32; inside a MFC application, gets a string with length 0.Inside this program I have a tool for handling this table but this cannot work because I cannot build up queries about rows with bugged URI: how can I fix this? Where this problem comes from? If you need more information please ask.
Thanks.
Looks like you have white spaces in the data and which is causing the issue and when you import data from CSV its most often happen.
So to fix it you may need to run the following update statement
update MyTable set URI = trim(URI);
The above will remove the white spaces from the column.
Also while importing data from CSV its better to use the TRIM() for the values before inserting into the database and this will avoid this kind of issues.