I have a database of analytic information.
my database as is below.
Table Name: display_page. here we store data about page when customer(of our user) create an order.(page has multiple sections). the table has 12 Million rows
element_impression. Here we store data about each section. (if the page has 5 sections we insert 5 rows for it. we have 10 unique sections and each will be repeated on page). the table has 45 Million rows
element_clicks. Here we store data about click over the section means when someone clicks on any section of the page we store the data into clicks table.
the table has 1.52 Million rows
Everything is worked fine since last year. But now it will create an issue when trying to fetch data according to the date range.
in future it will become 10X more from this stage. Please suggest with the best way to handle it.
Related
I have a dynamic table data as below
territory
01-May-2021
02-May-2021
INDIA
10
30
UAE
20
80
USA
300
10
Every day when we run the script a new column gets inserted with the current day.
I need this table data to be converted to HTML and send in EMAIL body to the respective recipients each day.
As this table is having dynamic data as date column each day kindly help.
I am using the sendmail -t function for triggering the mail from database.
This is too long for a comment.
Every day when we run the script a new column gets inserted with the current day.
This is broken. Are you aware that SQL Server has a limit on the number of columns? Are you aware that adding a new column requires rewriting all the existing data? Are you aware the by adding a new column, you cannot write queries?
What you should be doing is adding new rows. Each row should have (at least) three columns:
region
date
amount
Then adding new data is just adding rows to the table.
If you need data in the above format, you can use a view to structure the table you need. You can recreate the view every day -- when you are adding the new rows.
I have created the database in access, to enter daily inspection data, some time for one item we have to add multiple type rework entry and for that every time we have to select product name every time, and its time consuming
what I am trying to find that add multi-row in my form which will store different types to data in separate rows in the main table and product name will take automatically for all rows data
Person -> Item - > Work -> Component.
This are the main tables in the database.
I have to search for Item by a criteria. It will give a list. I will join the Person to get his "parent". After this maybe is a record in Work table maybe not, but if is, I will join also for Work I will join the list of Components if can be found.
The original code it was with nested tables. It will crash the browser, because taking to much memory with that design and is extremely slow around 150 records.
I rewrote it with divs the nested table. The performance got a huge boost, but start to be slow again because of buttons and design. ( It wasn't able to show 200 record before even after 10 min waiting!, now display 5k rows in 23 seconds)
Some of my benchmark logs:
SQL execution time: 0.18448090553284 seconds. Found 5624 rows.
For each result processing took: 0.29220700263977 seconds.
Writing the HTML code took:0.4107129573822 seconds.
Rows in HTML: 26551 headers + data.
Total Cells in HTML: **302491** headers and data.
Time until DOMready: 23691 milliseconds (in JavaScript)
0.18 + 0.29 + 0.41 = 0.88 So is around 1 second!
But when the browser actually want to show it to you ( paint ) it will take like 20 seconds!!!
Please don't suggest the paging! - the customer (final user) want to see all data in 1 web page for whatever his reason. No comment here.
Running on an i7 processor and 8/16 GB ram as requirement is accepted.
Most of the data rows has a collapse/expand button.
Most of the data row has the CRUD buttons: Add, Edit, Delete, View in details
All 4 kind of data table has headers and they don't match the length of the other kind of table header size, neither in column numbers.
When I want to just list the data in a blank page ( without design) and use 1 single table it is like 2 seconds or 3, not 20-30.
The original nested table solution has buttons with functionality in data row.
I would like to use it, and not implement it again.
My idea is to go back to the original nested table design ( to not re implement a lot of functionality of buttons) Then display only the top level table collapsed, with expand buttons. Then call an AJAX to get the second level data, when ready call the 3rd level then the 4th level.
The user is using intranet or the same PC as the server, so this maybe can be acceptable? -and doesn't have a blocking user interface for long time.
How would you handle this case, when there is not an option to show a next page button with 20 records / page.
I'm trying to make a master table in ms access that constantly updates the records of the unique model numbers based upon the most recent date.
The premade table was created by using a select query that brings all the records of multiple tables and inserts them into a single table. Within each table there are multiple fields however the only relevant fields are the Model number field and the Date. Across the tables the unique model numbers are repeated, but their individual time stamps are different.
I need the master table to select the records with the most recent date and place the whole record into the master table. I'm new to ms access and I don't know how I could do this. Any tips of suggestions would be most appreciated.
If you would need such a table then I'd say there's definitely something wrong with your database design.
Having said that, why don't you query on the most recent date of tables that hold Model Numbers?
It might help if you could elaborate on why you'd want to do this, as in why you have multiple tables that hold Model Numbers.
Maybe your current database design is open for improvement.
Kind regards,
Rene
I am loading a very big testing database into MYSQL (I have a script running to load all the tables). I think the loading process is going to take a few days so I am trying to determine which tables have been completely loaded and also make sure every tuple is correctly stored. I have many tables so I do not want to do a count for each table one by one; I was wondering if there is a function to display all the tables in my database with a summary of the information about them (ie number of rows). So far,I have found the function SHOW TABLES but this does not show any information about the tables.
Please let me know if there is a way to obtain this information without having to run queries for every table.
Thanks