How to join a group of tables with the same suffix? - mysql

So I am no MYSQL expert and I really need some help trying to figure this out. I currently have over 60 tables that I wish to join into a single table, none of the data in those tables match each other, so I need the rows of all the tables into a single one. They do have the same schema if that is the correct term, basically the same format. They all end in the same suffix '_dir'.
What I thought that could work was something like this,
Get all tables under the same suffix,
For each table in the table list join or insert row into main_table.
I don't know how to do this in mysql or if its even possible. I know I can use,
SELECT *
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME LIKE '%_dir%'
to get the list of all the tables, but how can I use this to iterate over every table?
Here is an example of input data:
table 1:
| NAME | INST_NAME | Drop
| data 1 | 'this is an example instance1 | 1.5
| data 1 | 'this is an example of instance2| 2.0
table 2:
| NAME | INST_NAME | DROP
| data 2 | 'this is an example instance1 | 3.0
| data 2 | 'this is an example of instance2| 4.0
Output table:
| NAME | INST_NAME | DROP
| data 1 | 'this is an example instance1 | 1.5
| data 1 | 'this is an example of instance2| 2.0
| data 2 | 'this is an example instance1 | 3.0
| data 2 | 'this is an example of instance2| 4.0
Note that I have to do this for over 60 tables not just 2. There are also other tables with different information in the same database, so I cant just join all tables in there.

You really need to fix your data structure. You should not be storing data in tables with the same structure -- that information should all go into a single table. Then you wouldn't have this issue.
For now, you can construct a view with all the data. You can generate the code for the view with something like this:
SELECT CONCAT('CREATE VIEW vw_dir AS',
GROUP_CONCAT(REPLACE('SELECT NAME, INST_NAME, `DROP` FROM [T]', '[T]'), TABLE_NAME)
SEPARATOR ' UNION ALL '
)
) as create_view_sql
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME LIKE '%_dir%';
Then take the SQL, run it, and you'll have a view called vw_dir. The next time you add a table, you'll need to drop the view and then recreate it.
With this solved, you can now start thinking about how to get all the data into a single table, without having the intermediate tables cluttering up your database.

Related

Find existence of a record in MySQL table with input data as a list

I have a list of ids in text format as a comma separated value like so
("12345", "12346", "12347", etc, etc)
I would like to find their existence or non existence from a table say devices table which has a column called device ids (not primary key)
Ideally i would like to get a list which says if each item exists or not.
So far I have tried to get the query of those that exist and I have to manually find the non existing ones.
Is there a for loop I have to run on stored procedures or something like that. Please help.
Table structure
<pre>
| id | device_id | device_name |
+------+-----------------+---------------+
| 71 | 352701060409650 | 57X |
| 13 | 352701060409700 | 582 |
</pre>
You need to create a query with left join to the same table with 'IFNULL' condition. There already has been a post for this topic. Please check this out here.

export phpList subscribers via sql in mysql database

For some reason, I am unable to export a table of subscribers from my phpList (ver. 3.0.6) admin pages. I've searched on the web, and several others have had this problem but no workarounds have been posted. As a workaround, I would like to query the mySQL database directly to retrieve a similar table of subscribers. But I need help with the SQL command. Note that I don't want to export or backup the mySQL database, I want to query it in the same way that the "export subscribers" button is supposed to do in the phpList admin pages.
In brief, I have two tables to query. The first table, user contains an ID and email for every subscriber. For example:
id | email
1 | e1#gmail.com
2 | e2#gmail.com
The second table, user_attribute contains a userid, attributeid, and value. Note in the example below that userid 1 has values for all three possible attributes, while userid's 2 and 3 are either missing one or more of the three attributeid's, or have blank values for some.
userid | attributeid | value
1 | 1 | 1
1 | 2 | 4
1 | 3 | 6
2 | 1 | 3
2 | 3 |
3 | 1 | 4
I would like to execute a SQL statement that would produce a row of output for each id/email that would look like this (using id 3 as an example):
id | email | attribute1 | attribute2 | attribute3
3 | e3#gmail.com | 4 | "" | "" |
Can someone suggest SQL query language that could accomplish this task?
A related query I would like to run is to find all id/email that do not have a value for attribute3. In the example above, this would be id's 2 and 3. Note that id 3 does not even have a blank value for attributeid3, it is simply missing.
Any help would be appreciated.
John
I know this is a very old post, but I just had to do the same thing. Here's the query I used. Note that you'll need to modify the query based on the custom attributes you have setup. You can see I had name, city and state as shown in the AS clauses below. You'll need to map those to the attribute id. Also, the state has a table of state names that I linked to. I excluded blacklisted (unsubscribed), more than 2 bounces and unconfirmed users.
SELECT
users.email,
(SELECT value
FROM `phplist_user_user_attribute` attrs
WHERE
attrs.userid = users.id and
attributeid=1
) AS name,
(SELECT value
FROM `phplist_user_user_attribute` attrs
WHERE
attrs.userid = users.id and
attributeid=3
) AS city,
(SELECT st.name
FROM `phplist_user_user_attribute` attrs
LEFT JOIN `phplist_listattr_state` st
ON attrs.value = st.id
WHERE
attrs.userid = users.id and
attributeid=4
) AS state
FROM
`phplist_user_user` users
WHERE
users.blacklisted=0 and
users.bouncecount<3 and
users.confirmed=1
;
I hope someone finds this helpful.

How to get the right "version" of a database entry?

Update: Question refined, I still need help!
I have the following table structure:
table reports:
ID | time | title | (extra columns)
1 | 1364762762 | xxx | ...
Multiple object tables that have the following structure
ID | objectID | time | title | (extra columns)
1 | 1 | 1222222222 | ... | ...
2 | 2 | 1333333333 | ... | ...
3 | 3 | 1444444444 | ... | ...
4 | 1 | 1555555555 | ... | ...
In the object tables, on an object update a new version with the same objectID is inserted, so that the old versions are still available. For example see the entries with objectID = 1
In the reports table, a report is inserted but never updated/edited.
What I want to be able to do is the following:
For each entry in my reports table, I want to be able to query the state of all objects, like they were, when the report was created.
For example lets look at the sample report above with ID 1. At the time it was created (see the time column), the current version of objectID 1 was the entry with ID 1 (entry ID 4 did not exist at that point).
ObjectID 2 also existed with it's current version with entry ID 2.
I am not sure how to achieve this.
I could use a query that selects the object versions by the time column:
SELECT *
FROM (
SELECT *
FROM objects
WHERE time < [reportTime]
ORDER BY time DESC
)
GROUP BY objectID
Lets not talk about the performance of this query, it is just to make clear what I want to do. My problem is the comparison of the time columns. I think this is no good way to make sure that I got the right object versions, because the system time may change "for any reason" and the time column would then have wrong data in it, which would lead to wrong results.
What would be another way to do so?
I thought about not using a time column for this, but instead a GLOBAL incremental value that I know the insertion order across the database tables.
If you are interting new versions of the object, and your problem is the time column(I assume you are using this column to sort which one is newer); I suggest you to use an auto-incremental ID column for the versions. Eventually, even if the time value is not reliable for you, the ID will be.Since it is always increasing. So higher ID, newer version.

SQL statement to return elements from a column only if no elements from a different column match

Sorry for the confusing question, I will try to clarify.
I have an SQL database ( that I did not create ) that I would like to write a query for. I know very little about SQL, so it is hard for me to even know what to search for to see if this question has already been asked, so sorry if it has. It should be an easy solution for those in the know.
The query I need is for a search I would like to perform on an existing data management system. I want to return all the documents that a given user has NOT signed-off on, as indicated by rows in a signoffs_table. The data is stored similarly to as follows: (this is actually a simplification of the actual schema and hides several LEFT JOINS and columns)
signoffs_table:
| id | user_id | document_id | signers_list |
The naive solution I had was to do something like the following:
SELECT document_id from signoffs_table WHERE (user_id <> $BobsID) AND signers_list LIKE "%Bob%";
This works if ONLY Bob signs the document. The problem is that if Bob and Mary have signed the document then the table looks like this:
signoffs_table:
-----------------------------------------------
| id | user_id | document_id | signers_list |
-----------------------------------------------
| 1 | 10 | 100 | "Bob,Mary,Jim" |
| 2 | 20 | 100 | "Bob,Mary,Jim" |
-----------------------------------------------
(assume Bob's ID = 10 and mary's ID = 20).
and then when I do the query then I get back document_id 100 (in row #2) because there is a row that Bob should have signed, but did not.
Is what I am trying to do possible with the given database structure? I can provide more details if needed. I am not sure how much details are needed.
I guess this query is what you mean:
SELECT document_id FROM signoffs_table AS t1
WHERE signers_list LIKE "%Bob%"
AND NOT EXISTS (
SELECT 1 FROM signoffs_table AS t2
WHERE (t2.user_id = $BobsID) AND t2.document_id = t1.document_id )
I believe your design is incorrect. You have a many-to-many relationship between documents and signers. You should have a junction table, something like:
ID DocumentID SignerID

Update one MySQL table with values from another

I'm trying to update one MySQL table based on information from another.
My original table looks like:
id | value
------------
1 | hello
2 | fortune
3 | my
4 | old
5 | friend
And the tobeupdated table looks like:
uniqueid | id | value
---------------------
1 | | something
2 | | anything
3 | | old
4 | | friend
5 | | fortune
I want to update id in tobeupdated with the id from original based on value (strings stored in VARCHAR(32) field).
The updated table will hopefully look like:
uniqueid | id | value
---------------------
1 | | something
2 | | anything
3 | 4 | old
4 | 5 | friend
5 | 2 | fortune
I have a query that works, but it's very slow:
UPDATE tobeupdated, original
SET tobeupdated.id = original.id
WHERE tobeupdated.value = original.value
This maxes out my CPU and eventually leads to a timeout with only a fraction of the updates performed (there are several thousand values to match). I know matching by value will be slow, but this is the only data I have to match them together.
Is there a better way to update values like this? I could create a third table for the merged results, if that would be faster?
I tried MySQL - How can I update a table with values from another table?, but it didn't really help. Any ideas?
UPDATE tobeupdated
INNER JOIN original ON (tobeupdated.value = original.value)
SET tobeupdated.id = original.id
That should do it, and really its doing exactly what yours is. However, I prefer 'JOIN' syntax for joins rather than multiple 'WHERE' conditions, I think its easier to read
As for running slow, how large are the tables? You should have indexes on tobeupdated.value and original.value
EDIT:
we can also simplify the query
UPDATE tobeupdated
INNER JOIN original USING (value)
SET tobeupdated.id = original.id
USING is shorthand when both tables of a join have an identical named key such as id. ie an equi-join - http://en.wikipedia.org/wiki/Join_(SQL)#Equi-join
It depends what is a use of those tables, but you might consider putting trigger on original table on insert and update. When insert or update is done, update the second table based on only one item from the original table. It will be quicker.