I know that sql can't save arrays (correct me if i'm wrong).
why?
I know this is a stupid question, but
Arrays are only structured data. Why can't sql save that?
Can i rewrite my mysql database or download a Addon for sql so i can save arrays?
Thanks in advance
Relational database management systems (RDBMS), such as MySQL, SQL Server, Oracle and PostgreSQL usually store data in tables. This is a very good way to store related data.
Let's say there are three entities: customers, orders, and products, and the orders contain multiple products. Four tables hence:
customers(customer_no, name)
products(product_no, name, price)
orders(order_no, customer_no, date)
order_details(order_no, product_no, amount)
We would provide indexes (i.e. search trees) to easily find orders of a customer or products in an order. Now let's say, we want to know how many orders have been made for product 123:
select count(distinct order_no)
from order_details
where product_no = 123;
The DBMS will quickly find the order_detail records for the product, because looking up an index is like searching by last name in a telephone book (binary search). And then it's mere counting. So only few records get read and the whole query is really fast.
Now the same with arrays. Something like:
products(product_no, name, price)
customers
(
customer_no,
name,
array of orders
(
order_no,
date,
array of products
(
product_no,
amount
)
)
)
Well, the order details are now hidden inside an order element which itself is inside a customer object. To get the number of orders for product 123, the only approach seems to be to read all customer records, loop through all orders and see whether they contain the product. This can take awfully long. Moreover without foreign key constraints for the relations between the entities, the arrays may contain product numbers that don't even exist.
Well, there may be ways to kind of index array data and there may be ways to guarantee data consistency for them, but the relational approach with tables has proven to solve these things extremely well. So we would avoid arrays and rather build our relations with tables instead. This is what a relational database is made for.
(Having said this, arrays may come in handy every now and then, e.g. in a recursive query were you want to remember which records have already been visited, but these occasions are rare.)
To answer my own question, i first want to thank for the comments
THANK YOU!
back to the question: ordinary sql cant save arrays and doesnt want to save save, because of normalization issues.
you can save arrays on another way:
A SQL Table is like an array. Link a new table as array. Create the table manually or, if the array could change, with Code. There is no need for arrays in sql
If you have to do, or want to do so, you can use Nosql or PostgreSql or save the Data with JSON, Oracle and XML
Related
I am building an order management system for an online store and would like to store information about the Product being ordered.
If I use a Foreign Key relationship to the Product, when someone changes the price, brand, supplier etc. of the Product or deletes it, the Order will be affected as well. I want the order management system to be able to display the state of the Product when it was ordered even if it is altered or deleted from the database afterwards.
I have thought about it long and hard and have come up with ideas such as storing a JSON string representation of the object; creating a duplicate Product whose foreign key I then use for the Order etc. However, I was wondering if there is a best practice or what other people use to handle this kind of situation in commercial software?
PS: I also have other slightly more complex situations, for instance, I would like the data for a User object attached to the Order to change as the User changes but then never get deleted when the User is deleted. An answer to the above question would definitely give me a good starting point.
This price-change problem is commonly handled in RDBMS (SQL) commerce applications by doing two things.
inserting rows into an order_detail table when an order is placed. Each row of that table contains the particulars of the item as sold: item_id, item_count, unit_price, total_price, unit_weight, total_weight, tax_status, and so forth. So, the app captures what actually was sold, and at what price. A later price change doesn't mess up sales records. You really have to do this.
a price table containing item_id, price, start_time, end_time. You retrieve the current price something like this:
SELECT item.item, price.price
FROM item
JOIN price ON item.item = price.item
AND price.start_date <= NOW()
AND (price.end_date > NOW() OR price.end_date IS NULL)
This approach allows you to keep track of historical prices, and also to set up future price changes. But you still copy the price into the order_detail table.
The point is: once you've accepted an order, its details cannot change in the future. You copy the actual customer data (name, shipping address, etc) into a separate order table from your current customer table when you accept the order, and (as mentioned above) the details of each item into an order_detail table.
Your auditors will hate you if you don't do this. Ask me how I know that sometime.
I would recommend creating attributes for the Order model and extracting the data you need one by one into those attributes while you are saving the model and then implementing a historical data table where you store JSONFields or some other version of the Product etc. when it is created or updated; that way people can refer to the historical data table if need be. This would be more efficient usage than storing the full fledged representation of the Product in the Order object as time taken to create the historical data is essentially charged to the admin creating the Product rather than the customer creating the Order. You can even create historical data objects in the background using threads etc. when you get to those advanced levels.
While it is hard answering your question without seeing your models.py at least, I will suggest archiving the results. You can add a boolean field called historical which defaults to False. When an order is made you need to set the previous order's (or orders') historical value to True in your view set or function.
Here, historical=True means the record is being archived. You can filter on this historical column to display what you want when. Sorry this is just a high-level outline.
i wanted to get your expert opinion about this dilema chosing bewteen JSON or Pivot Table
Let just say we have 2 tables here
people
jobs
A person may have multiple jobs, alas, a jobs might have multiple person subscirbed to it.
What is the best approach to it?
Method 1: JSON
I would have jobs column in people table, that contain json array of that person's jobs id, example : [1,2,4]
Method 2: Pivot
I would create pivot table job_person with job_id and person_id column, well, you know Laravel Eloquent style many to many pivot table
I have done some searching, and i found articels favouring each method, some say JSON better because it musch simpler, others would say Pivot is better due to that is how relationship database should work, etc etc.
But i want to know, which one should i use in what scenario? Like if it is just simple case like above scenario, JSON would be better?
What if there are other variables included like additional pivot columns
(Maybe each pivot also contain status column that can be set to active or past_job)
Or what if in the future we want to be able to get all peoples whom have a specific jobs, in which case Pivot would be preferable i think.
What if instead of jobs, the other table would be books and a person can have an extensive of books making we might have tens, or even hundreed pivot records just for one person? And there will be another hundreed persons?
What if instead of books, the other table were stocks in which case, a person might subscribed / unsubscribed multiple stock multiple times?
And maybe to the basic principle, what is each one's advantages/disadvantages?
Thank you very much
I would rather not choose JSON, as there's no benefit from choosing it, you will sacrifice many of the database features and make querying the data difficult and slow.
What if there are other variables included like additional pivot
columns (Maybe each pivot also contain status column that can be set
to active or past_job)
Job and Person are not dependent on each others, so you need to create an associative table between them something like "PersonJob" and add necessary information to it, this is easy to traverse in Laravel.
Or what if in the future we want to be able to get all peoples whom
have a specific jobs, in which case Pivot would be preferable i think.
You could easily query this using the associative table.
And maybe to the basic principle, what is each one's
advantages/disadvantages?
it just that relational databases are made for this kind of stuff and JSON offer no value just hardship.
I'm working on something that shows shops under a specific category, however I have an issue because I store the categories of a shop like this in a record with the id of a category. "1,5,12". Now, the problem is if I want to show shops with category 2, it "mistakens" 12 as category 2. This is the SQL right now.
SELECT * FROM shops WHERE shop_cats LIKE '%".$sqlid."%' LIMIT 8
Is there a way to split the record "shop_cats" by a comma in SQL, so it checks the full number? The only way I can think of is to get all the shops, and do it with PHP, but I don't like that as it will take too many resources.
This is a really, really bad way to store categories, for many reasons:
You are storing numbers as strings.
You cannot declare proper foreign key relationships.
A (normal) column in a table should have only one value.
SQL has poor string functions.
The resulting queries cannot take advantage of indexes.
The proper way to store this information in a database is using a junction table, with one row per shop and per category.
Sometimes, we are stuck with other people's really bad design decisions. If this is your case, then you can use FIND_IN_SET():
WHERE FIND_IN_SET($sqlid, shop_cats) > 0
But you should really fix the data structure.
If you can, the correct solution should be to normalize the table, i.e. have a separate row per category, not with commas.
If you can't, this should do the work:
SELECT * FROM shops WHERE CONCAT(',' , shop_cats , ',') LIKE '%,".$sqlid.",%' LIMIT 8
The table shops does not follow 1NF (1st Normal Form) i.e; every column should exactly one value. To avoid that you need to create another table called pivot table which relates two tables (or entities). But to answer your question, the below SQL query should do the trick.
SELECT * FROM shops WHERE concat(',',shop_cats,',') LIKE '%,".$sqlid.",%' LIMIT 8
I have 2 tables, users(~1000 users) and country(~50 countries). A user can support many countries so I am planning to create a mapping table, user_country. However, since I have 1000 users and 50 countries, I will have a maximum of 50000 entries for this table. Is this the best way to implement this or is there a more appropriate method for this?
If this is the best way, how can i add a user supporting many countries to this table using only one SQL statement?
For ex:
INSERT INTO user_country(userid, countrycode)
VALUES ('user001','US'),
('user001','PH'),
('user001','KR'),
('user001','JP')
Above SQL statement will be too long if a user supports all 50 countries. And I have to do it for 1000 users. Anyone have any ideas the most efficient way to implement this?
From the point of view of database design, a table like your user_country is the only sensible way to go. 50000 records are a breeze for MySQL, and having them together with the appropriate indexes will open up all ways of future use for those data.
As far as I can see, this is unrelated to the problem of many large SQL insert statements. No matter how you represent the data in the database, you will have to write statements containing, for each user, a list of countries.
This is a one-time action, right? So it doesn't need to be a masterpiece in software engineering. What I sometimes do is load the raw data in Excel, line by line, then write a formula that "calculates" the appropriate SQL statement for the first line, and copy this formula for all lines. Then throw all these statements at the database. Even if there are tens of thousands of them, it's not much effort.
Personally I'd do the insert based on a select:
INSERT INTO user_country SELECT 'user001', countryid from countries WHERE countryid IN ('US', 'PH', 'KR', 'JP');
You need to adapt to your column names.
The alternative of storing list of countries in a single column usercountries varchar (255) as US,FR,KR and so on would be possible as well - but you'd lose the ability to select users based on the country they support. In fact you don't lose it - but
SELECT * FROM users WHERE usercountries like '%KR%';
Is not a good query in terms of index usage. But as you only have 1000 users a tablescan will be mighty quick as well.
I want to try and keep this as one query and not use PHP, but it's proving to be tough.
I have a table called applications, that stores all the applications and some basic information about them.
Then, I have a table with all the types of applications in it, and that table contains a reference to another table which stores more specific data about the specific type of application in question.
select applications.id as appid, applications.category, type.title as type, type.id as tid, type.valuefld, type.tablename
from applications
left join type on applications.typeid=type.id
left join department on type.deptid=department.id
where not isnull(work_cat)
and work_cat != ''
and applications.deleted=0
and datei between '10-04-14' and '11-04-14'
order by type, work_cat
Now, in the old version, there is another query on every single result. Over hundreds of results... that sucks.
This is the query I'd like to integrate so I can get all the data in one result row. (Old is ASP, I'm re-writing it in PHP)
query = "select sum("&adors.fields("valuefld")&") as cost, description from "&adors.fields("tablename")&" where appid = '"&adors.fields("tablename")&"'"
Prepared statements, I'm aware, are the best solution, but for now they are not an option.
You can't do this with a plain SQL query - you need to have a defined set of tables that your query is based on. The fact that your current implementation queries from whatever table is named by tablename from the first result-set means that to get this all in one query, you will have to restructure your data. You have to know what tables you're querying from rather than having it dynamic.
If the reason for these different tables is the different information stored in each requiring different record (column) structures, you might want to look into Key/Value pair storage in a large table. Once you combine the dynamically named ones into a single location you can integrate your two queries together.