I am loading a very big testing database into MYSQL (I have a script running to load all the tables). I think the loading process is going to take a few days so I am trying to determine which tables have been completely loaded and also make sure every tuple is correctly stored. I have many tables so I do not want to do a count for each table one by one; I was wondering if there is a function to display all the tables in my database with a summary of the information about them (ie number of rows). So far,I have found the function SHOW TABLES but this does not show any information about the tables.
Please let me know if there is a way to obtain this information without having to run queries for every table.
Thanks
Related
I am building a web application that will run off of data that is produced for the public by a governmental agency. The issue is that the csv file that houses the data I need is a 2,000 column beast of a file. The file is what it is, I need to find the best way to take it and modify it. I know I need to break this data up into much smaller tables within MySQL, but I'm struggling with the best way to do this. I need to make this as easy as possible to replicate for next year when the data file is produced again (and every year after). I've searched for programs to help, and everything I've seen deals with a huge amount of rows, not columns. Has anyone else dealt with this problem before? Any ideas? I've spent the last week color coding columns in excel and moving data to new tabs, but this is time consuming, will be super difficult to replicate and I worry it leaves me open for copy and paste errors. I'm at a complete loss here!
Thank you in advance!
I suggest that you use functions in excel to give every column an automatic name "column1", "column2", "column3", etc.
After that import the entire csv file into MySQL.
Decide on which columns you want to group together into separate tables. This is the longest step and no program can help you manage this part.
Query your massive SQL table to get just the columns you want for each group. Export these queries to CSV and then import them as new tables in your database.
At the end, if you want, query all the columns you didn't put into separate groups. Make this a new table in the database and delete the original table to save on storage space.
Does this government csv file get updated and republished in the same format every time? If so you'll need to write a script to do all of the above automatically.
I have a table that contains data from repeated experiments (for example, site A has one sample, and the lab processed the sample three times obtaining slightly different values). I need to average these results in a separate table, but what I have read on the Microsoft support site is that a query that pulls data into another table with a calculated field is not possible on Access.
Can I query multiple data points from one table into a single calculated field in another table? Thank you.
UPDATE
I ended up doing a lot of manual adjustments of the file format to create a calculated field in the existing table that averages each sites data, so my problem is, for my current purposes, solved. However I would still like to understand. Following up with you both, I think the problem was that I had repeated non-unique IDs between rows when I probably should have made data columns with unique variable names so that I could query each variable name for an average.
So, instead of putting each site separately on the y axis, I formatted it by putting the sample number for each site on the x-axis:
I was able to at least create a calculated field using this second format in order to create an average value for each site.
Would have there been a way to write a query using the first method? Luckily, my data set was not at all very hefty, so I could handle a reformat manually, but if the case were with thousands of data entries, I couldn't have done that.
Also, here is the link to the site I mentioned originally https://support.office.com/en-ie/article/add-a-calculated-field-to-a-table-14a60733-2580-48c2-b402-6de54fafbde3.
Thanks all.
Good Morning,
I'm fairly new to Access VBA and I've been trying to find a solution to a problem:
I've created a form from which users upload an excel file to a database. File open prompt appears, user selects the file, temp table gets created and data gets pulled to this table. From there a set of macros populate the required fields and push the complete set to a perm table and then temp table gets deleted. Now I would like to take it a step further and try and count how many times a value has been uploaded to the table...
Lets say that the value appears in the table twice already, then if user tries to upload the same value for the third time it will be uploaded to a different table. Bear in mind that the file which users will upload may contain values that will be uploaded for the first, second, third, etc. time.
Do you have any suggestions or solutions to my problem? Is it even possible? If yes then how can I make Access to distinguish which records are being uploaded for the first, second, third, etc. time and follow appropriate paths?
I've been scouting the internet for several days now, but no one seems to have such issue.
Thank you in advance for replies.
I'm not sure I follow. You are essentially trying to prevent inserting duplicate data to a production table and if a duplicate is encountered at the record to a different table?
Let's say that we have one table with a field called sales_total and another table with a bunch of sales entries. Let's also, for a moment, imagine that it is impractical to count the entries every time we want to see the total number of sales.
Is it possible to have MySQL automatically update the sales_total field every time the number of sales entries changes?
I know that you can do this by running another query via C#, PHP or whatever - I'm just curious whether MySQL (or some other database system) can do this within itself?
P.S. This is of course a pretty banal example - the ideal solution should be able to handle more complex operations (storing several rows as a string in a field, etc).
use mysql trigger...
trigger on update from the first table should have some queries to update the second table.
I am trying to determine the best method of collecting a large list from a database and then displaying and filtering the results on the client side. Let me give a quick example:
Example: I've got a database with customer data and currently it contains around 2000 records. This number is constantly increasing. On my website I have a page that I want to be able to query said database based on information such as name, email, phone number etc. and of course display the results (when a user types in Smith it returns all records containing the name Smith). I am planning on using AJAX so that I can query the database and display the results on the fly similar to how google does it. When a user begins searching, results will start showing up on the page as they are found.
Possible Solutions:
Unfortunately I am stumped on how to go about implementing something like this. I am considering using a ValueList pattern. When the user first loads the page, should I be querying the database and storing every record in a collection and then searching that collection list and displaying the results on my jsp page? Essentially creating a java database. The thing I like about the ValueList pattern is that I take one huge hit on page load and dump the entire database in objects stored in a list. What if the database is larger though, say 2,000,000 records?
Or should I be using a simple DOA pattern without the ValueList and query the database for each individual search? This would result in a LOT of database queries, especially considering that I plan on returning results as the user types in the search box.
Edit: The more I think about this, the more it is an AJAX question. My biggest concern should be how to query my database while the user is typing. Do I set some sort of listener to listen for the user to stop typing and then perform the query?
I would use Solr for this type of task.
Fields, which you are going to use for searching should be indexed with Solr.
Then you do an ajax query to Solr and get the result. You can set the order, number of items per page and show results only for current page.
Solr has a lot of other features that can be useful for you.