I have been using an application I have made to distribute around my company which gives an average non-techie user(accountant, marketing type, mgmt) the ability to query any size DB with fast and friendly results. It uses the Dynamic.cs class to
Select all tables in a given DB
Select some filtered columns/fields in any Table
At runtime it figures out what the type is and then chooses which operators the user
can enter to aid their query
It gives the ability to only display the fields the user selects.
and Finally it gives the ability to Order and Group By
People, especially my superiors, LOVE it as it is incredibly useful. I can put this application on any DB and in 5 minutes they are able to query and export to Excel Worksheets in seconds.
Now, here is my issue, when I generate my select clause, if I have the field named "Object" I get a parse error from Dynamic.cs "Expecting a "(" or a "." - I am quite sure this is a keyword issue, and when the Parser hits the word Object it gets confused.**
One of my developers thinks, oh just write a partial class to get around the issue, but I think this is a serious enough bug, that I would like to fix the Dynamic.cs class -
Can anyone help me on this??? I have researched but have not found anything to point me in the right direction. I am pretty sure I could fix this but time is not on my side
Thanks in advance!!
First of all you are making a syntax error i guess, secondly even if you remove that syntax error it wont work because you are trying to create a dynamic select statement I guess, which is not as simple as you guess.
Dynamic linq library or regular expression can help you.
Related
I am using Sequelizer in my nodejs project and i couldn't make up anything from the official docs. Yet i managed to create tables and add rows etc. But the problem is that i need to query for objects with some special conditions on their associations.
I have a user model, a vote model and a Position model. A use can vote for an other user in a position. So this is the code that makes up the relations :
Position.hasMany(User,{as: 'Candidate'});
Votes.belongsToMany(User,{through: 'UserVote'});
User.belongsToMany(Votes,{through: 'UserVote'});
the problem is that i didn't find a way to query for the Positions which one known user didn't vote for yet. How do i do that ?
I am not 100% how to do that with associations but I will try to show you how I would do it using a regular query. Excuse the syntax if it isn't exactly perfect, this is just from memory. Assuming you have a user identifier provided (I'll call it userId).
Edit: I also noticed you are using MySQL. This works in Postgres but you might need to tweak the syntax for it to work for you. I don't know MySQL specifics but this should be close enough for you to get the idea!
Position.findAll({
where: [
`NOT EXISTS(
SELECT 1
FROM "votes"
WHERE "votes"."position_id" = "position"."id" AND
"votes"."user_id" = ${userId}
)`
]
}).then(...);
I have not found a better way to query existence yet myself. This pattern is usually my go-to for these kinds of queries. I made a few assumptions about your actual database model.
Also good to note that this is for Sequelize 3.x.x but I think it should more or less work on 4.x.x.
Good luck! :)
This is a super strange question, and it usefulness it's probably limited to my problem; I'm going to explain what I'm asking and why I need it.
My problem:
I have a table, let's say with 2 columns, serve the next table as example:
id|value
1 A
2 B
3 C
4 A
5 A
Now, If I do a "SELECT id WHERE value = 'A', I would get 3 results, 1, 4, 5. If I do a "SELECT id WHERE value = 'B', I would get 1 result, 2. And so on, if there were more entries, I would get the corresponding numbers of rows as my result according the value I'm looking in my query. It's all good.
But now, here comes my problem. Let's say I want to get every row for every query, but with the next restriction:
Do not modify the queries.
If I do "SELECT id WHERE value = 'A'", I would get every id, if I do "SELECT id WHERE value = 'B'", I would get every id, and so on.
"But if I can't modify my query, then what can I do?" You may ask, well, you can modify the table, like changing the value of the column 'value' to a value that would match every value, that's a wildcard, hence the title of the question, but I'm pretty sure if I update all 'value' values to '%', it doesn't work (I tried knowing this wouldn't work, but still, I couldn't lose anything trying).
So, you can do whatever you want, the only restriction is to not modify the queries.
I know this is kind of the inverse of how databases and tables should work, but this is a problem I've been presented with, maybe this is impossible, but maybe it's not.
Edit:
I know this makes little to no sense at all, but I'm asking this as a kind of challenge, appealing to the creatives minds out there. Don't worry about vulnerabilities or anything else, just ask yourselves: "How would I do it?"
Before I present any solutions, let me make it clear that you are solving the wrong problem. You should be figuring out how to change your queries; that restriction will continue to generate more problems. Any solution to this problem will be so complex it will generate more problems.
Hopefully this really is just an intellectual exercise.
I'm also going to only give sketches on how to do this, because this is just an intellectual exercise RIGHT?!
The first, and most comprehensive solution is to "just" change the source code of your MySQL database to respond to the queries however you like. It's an Open Source database. Download the source code, change it, recompile, and install.
The downside to this solution (assuming you can make it work) is it effects every connection to the database and has to be repeated every time you want to upgrade MySQL.
Assuming this is restricted to one table, and that the set of WHERE clauses is fixed, you can duplicate every row in that table to have every value which might be queried. For example, if you have id's 1 and 2 and value is only ever A, B or C, you'd make a table like this:
id|value
1 A
1 B
1 C
2 A
2 B
2 C
Then there are various man-in-the-middle attacks you can do to strip off the WHERE clause. If it's a fixed set of programs which are the problem you could alter the database API library they use. In Perl this would be the DBI library. In PHP this would be mysqli or PDO. And so on.
A more comprehensive solution would be to replace the MySQL server's socket (both the TCP and Unix socket) with your own little server. This would read and parse the MySQL network protocol (you may be able to extract the code to do this from the MySQL source), alter the query to strip the WHERE clause, and send it on to the real MySQL server.
These are all terrible solutions that are horribly difficult to implement correctly. Even if you got them working 100%, you're left with a system that does strange things to database queries which is likely to cause further problems down the road.
One of the most creative solutions to a problem is to realize you're solving the wrong problem.
I encourage you to post the circumstances that lead to this question, as another question, because that is the real problem. Also the management failures which lead to it will be a nice train wreck to watch.
Good people,
I have observed that the MS Access ORDER BY clause sorts records in non-ASCII way. This is different from MySQL - which is generally ASCII-compliant. Let me give you a little background so you understand why this is a problem to me.
Back in 2010, I wrote a generic database transaction logger. The goal was to detect changes occurring on (theoretically) any SQL database and log them in another database. To do this, I use a shadow MySQL database where I maintain a copy of the entire source database. The shadow database is designed using the EAV model so that it is agnostic to the source database schema.
Every once in a while, I read out of both the source and shadow databases, order the records based on their primary keys and format the records to correspond one-to-one. Then, I do a full database compare using a merge algorithm.
This solution has worked okay until last week when a user set it up against an Access database with string primary keys which are not always alphanumeric. All of a sudden the software started logging ghost transactions that have not happened on the source database.
On closer examination, I found out that MS Access orders non-alphanumeric characters in a fashion different from MySQL. As such, my merge algorithm, which assumes similar sort order for both source and shadow records, started to fail.
Now, I have figured out a way I could tweak my software to "cure" such primary keys before using them but it would help a great deal if I know precisely what is the nature of MS Access' ordering scheme. Any ideas will be highly appreciated.
PS: Let me know if there's anything I need to clarify. I am trying to avoid typing too much of what may not be useful.
I had a difficult time with this a few years ago. I'm sorry I didn't retain the solution, but it used VBA and it was not concise or elegant.
I opened the tables as DAO recordsets, advanced through the records, and used the strcomp() function to compare keys. I experimented a lot with the binary/text option of strcomp() and I believe it was finally necessary to insert an error-handling component!
This discussion may be relevant. Also this and this.
Hallo all.
I need to run the 'replace([column], [new], [old])' in a query executing on n Access 2003 DB. I know of all the equivalent stuff i could use in SQL, and believe me I would love to, but i don't have this option now. I'm trying to do a query where all the alpha chars are stripped out of a column ie. '(111) 111-1111' simply becomes '1111111111'. I can also write an awsum custom VBA function and execute the query using this, but once again, can't use these functions through JET. Any ideas?
Thanx for the replies guys. Ok let me clarify the situation. I'm running an .NET web application. This app uses an Access 2003 db. Im trying to do an upgrade where I incorporate a type of search page. This page executes a query like: SELECT * FROM [table] WHERE replace([telnumber], '-', '') LIKE '1234567890'. The problem is that there are many records in the [telnumber] column that has alpha chars in, for instance '(123) 123-1234'. This i need to filter out before i do the comparison. So the query using a built in VBA function executes fine when i run the query in a testing environment IN ACCESS, but when i run the query from my web app, it throws an exception stating something like "Replace function not found". Any ideas?
Based on the sample query from your comment, I wonder if it could be "good enough" to rewrite your match pattern using wildcards to account for the possible non-digit characters?
SELECT * FROM [table] WHERE telnumber LIKE '*123*456*7890'
Your question is a little unclear, but Access does allow you to use VBA functions in Queries. It is perfectly legal in Access to do this:
SELECT replace(mycolumn,'x','y') FROM myTable
It may not perform as well as a query without such functions embedded, but it will work.
Also, if it is a one off query and you don't have concerns about locking a bunch of rows from other users who are working in the system, you can also get away with just opening the table and doing a find and replace with Control-H.
As JohnFx already said, using VBA functions (no matter if built in or written by yourself) should work.
If you can't get it to work with the VBA function in the query (for whatever reason), maybe doing it all per code would be an option?
If it's a one-time action and/or not performance critical, you could just load the whole table in a Recordset, loop through it and do your replacing separately for each row.
EDIT:
Okay, it's a completely different thing when you query an Access database from a .net application.
In this case it's not possible to use any built-in or self-written VBA functions, because .net doesn't know them. No way.
So, what other options do we have?
If I understood you correctly, this is not a one-time action...you need to do this replacing stuff every time someone uses your search page, correct?
In this case I would do something completely different.
Even if doing the replace in the query would work, performance wise it's not the best option because it will likely slow down your database.
If you don't write that often to your database, but do a lot of reads (which seems to be the case according to your description), I would do the following:
Add a column "TelNumberSearch" to your table
Every time when you save a record, you save the phone number in the "TelNumber" column, and you do the replacing on the phone number and save the stripped number in the "TelNumberSearch" column
--> When you do a search, you already have the TelNumberSearch column with all the stripped numbers...no need to strip them again for every single search. And you still have the column with the original number (with alpha chars) for displaying purposes.
Of course you need to fill the new column once, but this is a one-time action, so looping through the records and doing a separate replace for each one would be okay in this case.
I find that when trying to construct complex MySQL joins and groups between many tables I usually run into strife and have to spend a lot of 'trial and error' time to get the result I want.
I was wondering how other people approach the problems. Do you isolate the smaller blocks of data at the end of the branches and get these working first? Or do you start with what you want to return and just start linking tables on as you need them?
Also wondering if there are any good books or sites about approaching the problem.
I don't work in mySQL but I do frequently write extremely complex SQL and here's how I approach it.
First, there is no substitute whatsoever for thoroughly understanding your database structure.
Next I try to break up the task into chunks.
For instance, suppose I'm writing a report concerning the details of a meeting (the company I work for does meeting planning). I will need to know the meeting name and sales rep, the meeting venue and dates, the people who attened and the speaker information.
First I determine which of the tables will have the information for each field in the report. Now I know what I will have to join together, but not exactly how as yet.
So first I write a query to get the meetings I want. This is the basis for all the rest of the report, so I start there. Now the rest of the report can probably be done in any order although I prefer to work through the parts that should have one-one relationshisps first, so next I'll add the joins and the fields that will get me all the sales rep associated information.
Suppose I only want one rep per meeting (if there are multiple reps, I only want the main one) so I check to make sure that I'm still returning the same number of records as when I just had meeting information. If not I look at my joins and decide which one is giving me more records than I need. In this case it might be the address table as we are storing multiple address for the rep. I then adjust the query to get only one. This may be easy (you may have a field that indicates the specific unique address you want and so only need to add a where condition) or you may need to do some grouping and aggregate functions to get what you want.
Then I go on to the next chunk (working first through all the chunks that should have a 1-1 relationshisp to the central data in this case the meeting). Runthe query nd check the data after each addition.
Finally I move to those records which might have a one-many relationship and add them. Again I run the query and check the data. For instance, I might check the raw data for a particular meeting and make sure what my query is returning is exactly what I expect to see.
Suppose in one of these additions of a join I find the number of distinct meetings has dropped. Oops, then there is no data in one of the tables I just added and I need to change that to a left join.
Another time I may find too many records returned. Then I look to see if my where clause needs to have more filtering info or if I need to use an aggreagte function to get the data I need. Sometimes I will add other fields to the report temporarily to see if I can see what is causing the duplicated data. This helps me know what needs to be adjusted.
The real key is to work slowly, understand your data model and check the data after every new chunk is added to make sure it is returning the results the way you think they should be.
Sometimes, If I'm returning a lot of data, I will temporarily put an additonal where clause on the query to restrict to a few items I can easily check. I also strongly suggest the use of order by because it will help you see if you are getting duplicated records.
Well the best approach to break down your MySQL query is to run the EXPLAIN command as well as looking at the MySQL documentation for Optimization with the EXPLAIN command.
MySQL provides some great free GUI tools as well, the MySQL Query Browser is what you need to use.
When running the EXPLAIN command this will break down how MySQL interprets your query and displays the complexity. It might take some time to decode the output but thats another question in itself.
As for a good book I would recommend: High Performance MySQL: Optimization, Backups, Replication, and More
I haven't used them myself so can't comment on their effectiveness, but perhaps a GUI based query builder such as dbForge or Code Factory might help?
And while the use of Venn diagrams to think about MySQL joins doesn't necessarily help with the SQL, they can help visualise the data you are trying to pull back (see Jeff Atwood's post).