I found a weard problem with my MySQL DB.
sometime when I insert new data into it, the way it arranges the data is like a stack, for example
4 (newest)
3
2
1 (oldest)
...
how can I make it arranged like this?
1 (newest)
2
3
4 (oldest)
thanks all.
SELECT *
FROM TABLE
ORDER BY ID
You have to remember that when viewing/selecting data from a table without any ORDER BY specified does not garuantee any specific order.
The way you are view the data (unordered) can be due to any one of a lot of factos (the database engine, schema, page storage, page fragmentation, indexes, primary keys, or simply execution plan optimization).
The SQL standards specifically states that tables do not have a "natural" order. Therefore, the database engine is free to return a request without an ORDER BY in any order it wants to. The order may change from one request to another because most engines choose to return the data in whatever order they can get it to you the most rapidly.
It follows, therefore, that if you want the data out in a particular order you must include a column in your table whose job is to proxy for the order in which you added records to the table. Two common ways of doing this are using an autoincrement field which will be in numerical order from oldest to newest record, and a TIMESTAMP column which does just what it says. Once you have such a column you can use ORDER BY ColumnName when searching to get an ordered result set.
Related
I imported the 3 tables in a file to my database. Worked perfectly.
Here's the file code: https://pastebin.com/EVr9qGxe
....
However, I want to change the variable in each table so I went to phpMyAdmin to change the variables in the tables.
They change just fine, however, the options appear in alphabetical, and not the order I entered them in or the country_id
for example if I entered
USA with country_id 1
France with country_id 2
France appears first in the list.
So my question is, what should I change in the code linked above in order to remove the alphabetical auto-order and make the options appear by their ID?
If I understand correctly, your question is similar to this one. There's no way of permanently changing this behavior I'm aware of. You could add
ALTER TABLE `your_table` ORDER BY `column_name`
But the table will 're-order' itself back to where it was on the next CRUD operation performed on that table, and for some databases this query simply won't work.
As you are using phpMyAdmin, there's a simpler solution without coding- just enter your database, choose your table, and click on any column. Clicking on any column will order your table by values from that column. Clicking it again will change ordering from ascending to descending.
Rows in relational database are not really ordered, but you can order them when needed, by performing queries (with use of ORDER BY). For example, if you need to select all rows from table 'users', and order them by 'creation_date', you'd simply use SELECT * FROM users ORDER BY creation_date, but it won't change rows ordering in the table itself- you'd just get ordered result displayed.
We are building an analytics engine which has to store attribute preference score for each user. We are expecting 400 attributes and they may change(at what frequency is not known as yet). We are planning to store this in Redshift.
My qs is:
Should we store as 1 row per user with 400 cols(1 column for each attribute)
or should we go for a table structure like
(uid, attribute id, attribute value, preference score) which will be (20-400)rows by 3 columns
Which kind of storage would lead to a better performance in Redshift.
Should be really consider NoSQL for this?
Note:
1. This is a backend for real time application with increasing number of users.
2. For processing, the above table has to be read with entire information of all attibutes for one user i.e indirectly create a 1*400 matrix at runtime.
Please help me which desgin would be ideal for such a use case. Thank you
You can go for tables like given in this example and then use bitwise functions
http://docs.aws.amazon.com/redshift/latest/dg/r_bitwise_examples.html
Bitwise functions are here
For your problem, I would suggest a two table design. Its more pain in the beginning but will help in future.
First table would be a key value kind of first table, which would store all the base data and would be kind of future proof, where you can add/remove more attributes, but this table will continue working.
And a N(400 in your case) column 2nd table. This second table you can build using the first table. For the second table, you can start with a bare minimum set of columns .. lets say only 50 out of those 400. So that querying this table would be really fast. And the structure of this table can be refreshed periodically to match with the current reporting requirements. Also you will always have the base table in case you need to backfill any data.
I have a table of > 250k rows of 'names' (and ancillary info) which I am displaying using jQuery Datatables.
My Users can choose any 'name' (Row), which is then flagged as 'taken' (and timestamped).
A (very) cut down version of the table is:
Key, Name, Taken, Timestamp
I would like to be able to display the 'taken' rows (in timestamp order) first and then the untaken records in their key order [ASC] next.
The problem would be simple, but, because of size constraints (both visual UI & data set size) My display mechanism paginates - 10 / 20 / 50 / 100 rows (user choice)
Which means a) the total number of 'taken' will vary and b) the pagination length varies.
Thus I can see no obvious method of keeping track of the pagination.
(My Datatable tells me the count of the start record and the length of the displayed records)
My SQL (MySQL) at this level is weak, and I have no idea how to return a record set that accounts for the 'taken' offset without some kind of new (or internal MySQL) numeric indices to paginate to.
I thought of:
Creating a temporary table with the key and a new numeric indices on
each pagination.
Creating a trigger that re-ordered the table when the row was
'taken'.
Having a "Running order" column that was updated on each new 'taken'
Some sort of cursor based procedure (at this point my hair was
ruffled as the explanations shot straight over the top of my head!)
All seem excessive.
I also thought of doing a lot of manipulation in PHP (involving separate queries, dependant on the pagination size, amount of names already taken, and keeping a running record of the pagination position.)
To the Human Computer (Brain) the problem is untaxing - but translating it into SQL has foxed me, as has coming up with a fast alternative to 1-3 (the test case on updating the "Running order" solution took almost three minutes to complete!)
It 'feels' like there should be a smart SQL query answer to this, but all efforts with ORDER BY, LIMITS, and the like fall over unless I return the whole dataset and do a lot of nasty counting.
Is there something like a big elephant in the room I am missing - or am I stuck with the hard slog to get what I need.
A query that displays the 'taken' rows (in timestamp order) first and then the untaken records in their key order [ASC] next:
SELECT *
FROM `table_name`
ORDER BY `taken` DESC, IF(`taken` = 1, `Timestamp`, `Key`) ASC
LIMIT 50, 10
The LIMIT values: 10 is the page size, 50 is the index of the first element on page 6.
Change the condition on IF(taken = 1,Timestamp,Key) with the correct condition to match the values you store in column taken. I assumed you store 1 when the row is 'taken' and 0 otherwise.
I am new to MySQL partitioning, therefore any example will be appreciated.
I am trying to create a sort of an ageing mechanism for a data that is distributed between several MyISAM tables.
My question will actually include several sub-questions.
The relevant tables are:
First table contains raw data with high input frequency (next to each record there is an auto incremented id).
Second table contains processed results, there is a result record per every raw data record (result record contains the source id record of the auto incremented field of raw data record)
Questions:
I need to be able to partition the raw data table and result data table similarly so that both of them will include only 10 weeks of data in single partition (each raw data record contains unixtimestamp field), how do i do it , can someone write small example case for two such tables?.
I want to be able to change the 10 weeks constraint on the fly.
I want that when ever the current partition will be filled or a new partition is created , the previous (10 weeks before) partition will be deleted automatically.
I don't want the auto increment id integer to be overflown, as much as i understand the ids are unique for the partition only, so if i am not wrong the auto increment id will start from zero for the next partition? but what if the previous partition still exist, will i have 2 duplicated ids , how i know to reference only for the last id when i present a result record?
I want to load raw data using LOAD DATA INTO... instead of multiple inserts , is MySQL partitioning functionality affected?
And the last question, would you suggest some other approach to implement aging mechanism (i am writing Java implementation product that processes around 1 GB or raw data per day and stores the results in MySQL)
It's hard to give a real answer on this question since it depends on your data. But let me give you some things to think about.
I assume we're talking about some kind of logs with recent data (so not spanning multiple years). You can partition by range. You could add one field to your table with the year/week number (ie 201201, 201202, etc). If this question is related to your question about importing into multiple tables, you can easily do this is that import script.
On the fly as in, repartition your data on the fly (70GB?). I would not recommend it. But you could do it if you had the weeknumber in there. If you later want to change it to 12 days, you could add a column for the date and partition by that.
Well it won't be deleted automatically but a cron job can handle that right? Just check how many partitions there are, and if there are 3(?) delete the first one.
The partition needs to have a primary index on the field that you partition (if you want to use auto increment). Therefor you can never fully rely on the auto increment id alone. I don't see a way around this.
I'm not sure what you mean.
If your data is just some logs in chronological order then you might just use separate tables for each period. Then before you start the new period (at 00:00) check the last id of the last table, create a new table and set the auto increment to that value +1. Then your import will decide when a new period will begin so it can be easily changed. Your import script can use a small table in where it can store the next period.
LOAD DATA is really quite fast. I would just have two steps(in no partic order) - LOAD DATA and then 'delete .. where date < 10 weeks'. Autoincrement will go on for as long as the datatype you're using. If you wanted to be super careful you could push it back to zero periodically.
Once the data is in the 'raw' table run your routine to create the 'processed' table. We use a v similar process where I work. We keep a separate table that has 'write' and 'parse' pointers to all of our 'raw' tables. As new data comes in and gets parsed the appropriate row pointers get set. If the 'raw' table gets truncated you can reset the 'write' pointer but leave the 'parse' pointer. (we store the offset in another table when this happens - just to be sure).
And if I recommend , creating the index column for each of the related columns can also enhanced the performance Delete old data from multiple related tables since we have just compared the index numbers rather than strings.
I wonder if your tables are being sorted or not.
I want to store some records in mysql database. records have a date column. i want to store them in sorting order of that date column.
For example, record having date 27/sep/2011 get stored as first row on the top of record having date 26/sep/2011 as:
id_1,name_1,27/sep/2011
id_2,name_2,26/sep/2011
if new records come on future dates they would get inserted on the top.
I DONT want to order them while using select by using order by desc .
i want they get inserted into db directly in sorted order.
how to do this???
thanks...
I am always surprised when people want to determine physical order of storing records.
Basically, it's a terrible idea for multiple reasons.
1) How the record is physically stored should not be of your concern.
2) How the record is presented should be of your concern. That's why we have ORDER BY built in.
3) Determining physical storage should be done by experts in the field, since it has performance implications - which is a topic in its own and I won't go into details.
Basically, worry about getting the data out in the sorted order, not getting it in in the sorted order.
Reason why it's a bad idea is because you'll be tampering with the primary key which is never, ever a good idea. On top of that, you'll have to reorder the records every time you insert something. Just don't reinvent hot water.
You could do this by adding another table - inserting all of the records into that table (the current and the new ones) then doing and insert as follows :
INSERT into newtable
select * from temptable
order by temptable.date
Why do you need to do this ? why not just use orderby on the query ?
As pointed out in the comments below - you would need to truncate the newtable each time
You cannot choose where to insert your row.
Here's one possible solution: MySQL syntax for inserting a new row in middle rows?