I'm working with a MySQL table through MySQL Workbench. Currently, many of my tables are appearing more than once with some appearing 10+ times. It is not messing anything up with my application or when I run a query on any of these tables, but something that I feel should be fixed. Haven't been able to find anything and just assuming this is a quick fix.
MySQL Table List
Currently running MySQL Workbench 8.0.23 build 365764 SE (64 bits), MySQL Azure Database System.
Also, is there any reason that this would be happening? I have a variety of php files set up to read/write data, my hunch is that some lines of code may be creating duplicates that I need to nail down, is there any specific SQL syntax that would be creating these duplicates (other than CREATE TABLE..., I do not have anything of that nature set to run).
Thanks for the help.
Related
A customer notifies me that the search of an application we have developed for him is not working properly. The search is based on queries on Full-Text Search indexes on InnoDB tables in MySQL 5.6 (the first version that supported it).
When performing the query manually I check that it only returns 1 result when it should return 2. I tried with other search terms and the same table row is always omitted. The same queries using LIKE works well.
When trying to replicate the problem in a virtualized environment with the same OS and MySQL server versions I am unable to reproduce it, as the query works correctly after restoring a database dump.
After trying several options what has worked is to run OPTIMIZE TABLE. The search query works well after optimizing the table, showing the missing record.
Why does this happen? What is the explanation for this problem and how can it be detected or prevented?
i dont know much about your code, your database, your queries, but i see only 2 situations :
a row level locking which is never released : (may be a bug in mysql) https://dev.mysql.com/doc/refman/8.0/en/innodb-locking.html
table corruption : i'm not sure if OPTIMIZE TABLE is repairing anything or only reducing fragmentation.
I have a MySQL table with 650 columns and 1 row. I know this is a bad design (which I inherited) and it WILL be moved to a simple two-column layout, but at the moment I'm trying to diagnose a different problem to be able to understand it. I don't want to fix the design issues if it will mask the system configuration issue I'm currently facing.
Background: I have a system which takes the following steps in building an instance of a web application:
Use a skeleton.sql file to build the "version 0" of the database.
Run an upgrade.sh script which looks through an "upgrades" directory and...
...runs each of those upgrade files to change the database structure (including adding new columns to the already-too-wide table), in proper order.
The issue is, on my localhost server (WAMP) while running one of the scripts in step 3, I'm getting an error when attempting to add more columns to that table:
Row size too large. The maximum row size for the used table type, not counting BLOBs, is 8126.
Again, I know the table is too "wide" in theory, but I don't know any query I can run to calculate how wide it is. Further, I'm not getting this error on my RDS server when I run this, and since I don't know how to get the "width" of the table I don't know how to compare (though given localhost's VARCHARs are populated with FEWER characters, I expect it to be smaller regardless).
Everything I'm reading on StackOverflow says that all tables have the same "width" limit, regardless of engine. I suspect it's a configuration issue, but if this is a hard limit, why is it working on my RDS server but not localhost?
My only thought was character code differences? Have you looked into that? (Sorry can't comment too new a user).
I need to start off by pointing out that by no means am I a database expert in any way. I do know how to get around to programming applications in several languages that require database backends, and am relatively familiar with MySQL, Microsoft SQL Server and now MEMSQL - but again, not an expert at databases so your input is very much appreciated.
I have been working on developing an application that has to cross reference several different tables. One very simple example of an issue I recently had, is I have to:
On a daily basis, pull down 600K to 1M records into a temporary table.
Compare what has changed between this new data pull and the old one. Record that information on a separate table.
Repopulate the table with the new records.
Running #2 is a query similar to:
SELECT * FROM (NEW TABLE) LEFT JOIN (OLD TABLE) ON (JOINED FIELD) WHERE (OLD TABLE.FIELD) IS NULL
In this case, I'm comparing the two tables on a given field and then pulling the information of what has changed.
In MySQL (v5.6.26, x64), my query times out. I'm running 4 vCPUs and 8 GB of RAM but note that the rest of my configuration is default configuration (did not tweak any parameters).
In MEMSQL (v5.5.8, x64), my query runs in about 3 seconds on the first try. I'm running the exact same virtual server configuration with 4 vCPUs and 8 GB of RAM, also note that the rest of my configuration is default configuration (did not tweak any parameters).
Also, in MEMSQL, I am running a single node configuration. Same thing for MySQL.
I love the fact that using MEMSQL allowed me to continue developing my project, and I'm coming across even bigger cross-table calculation queries and views that I can run that are running fantastically on MEMSQL... but, in an ideal world, i'd use MySQL. I've already come across the fact that I need to use a different set of tools to manage my instance (i.e.: MySQL Workbench works relatively well with a MEMSQL server but I actually need to build views and tables using the open source SQL Workbench and the mysql java adapter. Same thing for using the Visual Studio MySQL connector, works, but can be painful at times, for some reason I can add queries but can't add table adapters)... sorry, I'll submit a separate question for that :)
Considering both virtual machines are exactly the same configuration, and SSD backed, can anyone give me any recommendations on how to tweak my MySQL instance to run big queries like the one above on MySQL? I understand I can also create an in-memory database but I've read there might be some persistence issues with doing that, not sure.
Thank you!
The most likely reason this happens is because you don't have index on your joined field in one or both tables. According to this article:
https://www.percona.com/blog/2012/04/04/join-optimizations-in-mysql-5-6-and-mariadb-5-5/
Vanilla MySQL only supports nested loop joins, that require the index to perform well (otherwise they take quadratic time).
Both MemSQL and MariaDB support so-called hash join, which does not require you to have indexes on the tables, but consumes more memory. Since your dataset is negligibly small for modern RAM sizes, that extra memory overhead is not noticed in your case.
So all you need to do to address the issue is to add indexes on joined field in both tables.
Also, please describe the issues you are facing with the open source tools when connect to MemSQL in a separate question, or at chat.memsql.com, so that we can fix it in the next version (I work for MemSQL, and compatibility with MySQL tools is one of the priorities for us).
i have a running 5.1.47 versioned slave, which i upgraded to 5.5.8 almost without anyproblem.
Except one. The thing is, there is that index cause i can not create same job index with same name but with different name. But somehow mysql doesnot get it.
I am stuck. and keep getting this message on mysqld.log.
For now i added another index with different name doing same job and user of slave is going good for now. But i need to fix this so i can upgrade other slave too.
Error table x/users contaions 10 indexes inside Innodb, which is different from the number of indexes 9 defined in mysql.
Anyone face such similar problem?
thanks.
According to Google Translate of this blog entry (which n.b. isn't great so I might've misread it) you might want do dump the whole database and import it back. This way the database will rebuild all indexes and it might just do the trick.
i solved that way. But when i export from 5.5.8 and index was not in mysqldump file, so i add manually and it worked. But there was data inconsistency 6 records was mismatch.
The odd thing is, it doesnot use index, it doesnot let you recreate one with the same name, and its not on show create table list, but its there, same index can be created via different name but not that one and that one is not listed at all.
Somehow after import 6 records was different then other tables after start slave again.
So for inconsistency i start from all over and it worked.
I did, copy data portion of a slave to 5.5.8 side and then start and mysql_upgrade it again.
And this time i had no problem at all. I dont know how come it works but it worked.
Let's say we have "system A" comprising a MySQL database, with several tables.
After a while you want to optimize system A by removing any unused tables and/or columns, system A being quite large and difficult to overlook. Is there a tool or method that lets you run the system for a while, and then do an analysis which prints out general use of tables, columns etc - which would make it possible to find unused tables and columns.
I'm thinking of maybe hooking into the system, logging all SQL commands going to the server, but would in that case rather be doing that on the database side, rather than the application side.
The General Query Log is probably what you're looking for.
According to MySQL, with the General Query Log on:
The server writes information to this log when clients connect or disconnect, and it logs each SQL statement received from clients.
You need to start MySQL with the --log[=file_name] or -l [file_name] option in order to use it.
Assuming you are already linking the database with an application of some sort, it shouldn't then be hard to find the columns that are being used over the columns that are not.
Further a program such as profiler (not sure on the MySQL equivalence) can be used to display all the SQL calls. From this you will know all the columns that are being used.