I have a database table at the backend, that provides various items for sale and their prices as well. For the sake of simplicity, let us assume that the schema for the same is like so:-
class Product(models.Model):
item_id = models.Integer()
item_name = models.CharField(max_length=30)
item_price = models.CharField(max_length=30)
item_count = models.Integer()
There is an API getProducts(), that is used to show to the enduser all the products available, including their price.
There is another API buyProduct(product_id), exposed to the enduser, using which he may purchase the item after seeing its price. There is a third API, updatePrice(product_id, new_price) available to the administration backend using which the price may be modified.
Now w/o having a database transaction that spans across >1 HTTP requests, how can I ensure that the API buyProduct() succeeds only if the price is the same as what the enduser saw (basically whatever getProducts() had returned last to the browser). It is possible that a customer sees the price of a product as 1$, and clicks BUY, but simultaneously the price at the backend was changed to 2$.
I have two solutions:-
Fix the buyProduct() API to include a price, which indicates
whichever price that was shown to the enduser. If the product price
has changed, throw an error.
Include a GUID against the products which represents the version of
the product row. This is then carried all the way to the enduser's
browser(but not shown to the user), and quoted in the buyProduct()
also. Anytime there is a modification to any row in the Products
table, the GUID will get updated. buyProduct() will ensure that at
the time the API is executed, the product's GUID should be what the
API carried.
I believe <2> is the way to go, since even the item_count is prone to change. But is this how everyone does things? Is there a django pattern to solve this?
Just realised that the E-Tag header and the Cache-Control headers in HTTP seem to be a recommended way to avoid data corruption. But do folks really follow this way? I found a link about this here
Related
I am building an order management system for an online store and would like to store information about the Product being ordered.
If I use a Foreign Key relationship to the Product, when someone changes the price, brand, supplier etc. of the Product or deletes it, the Order will be affected as well. I want the order management system to be able to display the state of the Product when it was ordered even if it is altered or deleted from the database afterwards.
I have thought about it long and hard and have come up with ideas such as storing a JSON string representation of the object; creating a duplicate Product whose foreign key I then use for the Order etc. However, I was wondering if there is a best practice or what other people use to handle this kind of situation in commercial software?
PS: I also have other slightly more complex situations, for instance, I would like the data for a User object attached to the Order to change as the User changes but then never get deleted when the User is deleted. An answer to the above question would definitely give me a good starting point.
This price-change problem is commonly handled in RDBMS (SQL) commerce applications by doing two things.
inserting rows into an order_detail table when an order is placed. Each row of that table contains the particulars of the item as sold: item_id, item_count, unit_price, total_price, unit_weight, total_weight, tax_status, and so forth. So, the app captures what actually was sold, and at what price. A later price change doesn't mess up sales records. You really have to do this.
a price table containing item_id, price, start_time, end_time. You retrieve the current price something like this:
SELECT item.item, price.price
FROM item
JOIN price ON item.item = price.item
AND price.start_date <= NOW()
AND (price.end_date > NOW() OR price.end_date IS NULL)
This approach allows you to keep track of historical prices, and also to set up future price changes. But you still copy the price into the order_detail table.
The point is: once you've accepted an order, its details cannot change in the future. You copy the actual customer data (name, shipping address, etc) into a separate order table from your current customer table when you accept the order, and (as mentioned above) the details of each item into an order_detail table.
Your auditors will hate you if you don't do this. Ask me how I know that sometime.
I would recommend creating attributes for the Order model and extracting the data you need one by one into those attributes while you are saving the model and then implementing a historical data table where you store JSONFields or some other version of the Product etc. when it is created or updated; that way people can refer to the historical data table if need be. This would be more efficient usage than storing the full fledged representation of the Product in the Order object as time taken to create the historical data is essentially charged to the admin creating the Product rather than the customer creating the Order. You can even create historical data objects in the background using threads etc. when you get to those advanced levels.
While it is hard answering your question without seeing your models.py at least, I will suggest archiving the results. You can add a boolean field called historical which defaults to False. When an order is made you need to set the previous order's (or orders') historical value to True in your view set or function.
Here, historical=True means the record is being archived. You can filter on this historical column to display what you want when. Sorry this is just a high-level outline.
I am having a legacy system with the MySQL at the backend and python as the primary programming language.
Recently we have a scenario where we need to display a dashboard with the information present in the MySQL database. The data in the table changes every second.
This can be thought of similar to a bid application where people bid constantly. Every time a user bids a record goes in to the database. When an user updates his bid it updates the previous value.
I also have few clients who monitor this dashboard which updates the statistics.
I need to order this data in realtime as people bid in real time.
I don't prefer to run queries against MySQL because at any second I may have few 1000 clients querying the database. This will create load on database.
Please advice.
If you need to collect and order data in realtime you should be looking at the atomic ordered map and ordered list operations in Aerospike.
I have examples of using KV-ordered maps at rbotzer/aerospike-cdt-examples.
You could use a similar approach with the user's ID being the key, the bid being a list with the structure [1343, { foo: bar, ts: 1234, this: that} ]. The bid amount in cents (integer) is the first element of the list, all the other information is in a map in the second element position.
This will allow you to update a user bid with a single map operation, get back the user's bid with a single operation, order by rank (on the bid amount) to get the top bids ordered, get all the bids in a particular range, etc. You would have one record per item, with all the bids in this KV-sorted map.
I am developing an app with PhoneGap and have been storing the user id and user level in local storage, for example:
window.localStorage["userid"] = "20";
This populates once the user has logged in to the app. This is then used in ajax requests to pull in their information and things related to their account (some of it quite private). The app is also been used in web browser as I am using the exact same code for the web. Is there a way this can be manipulated? For example user changes the value of it in order to get info back that isnt theirs?
If, for example another app in their browser stores the same key "userid" it will overwrite and then they will get someone elses data back in my app.
How can this be prevented?
Before go further attack vectors, storing these kind of sensitive data on client side is not good idea. Use token instead of that because every single data that stored in client side can be spoofed by attackers.
Your considers are right. Possible attack vector could be related to Insecure Direct Object Reference. Let me show one example.
You are storing userID client side which means you can not trust that data anymore.
window.localStorage["userid"] = "20";
Hackers can change that value to anything they want. Probably they will changed it to less value than 20. Because most common use cases shows that 20 is coming from column that configured as auto increment. Which means there should be valid user who have userid is 19, or 18 or less.
Let me assume that your application has a module for getting products by userid. Therefore backend query should be similar like following one.
SELECT * FROM products FROM owner_id = 20
When hackers changed that values to something else. They will managed to get data that belongs to someone else. Also they could have chance to remove/update data that belongs to someone else agains.
Possible malicious attack vectors are really depends on your application and features. As I said before you need to figure this out and do not expose sensitive data like userID.
Using token instead of userID is going solved that possible break attemps. Only things you need to do is create one more columns and named as "token" and use it instead of userid. ( Don't forget to generate long and unpredictable token values )
SELECT * FROM products FROM owner_id = iZB87RVLeWhNYNv7RV213LeWxuwiX7RVLeW12
The title is somewhat hard to understand, so here is the explanation:
I am building a system, that deals with retail transactions. Meaning - purchases. I have a database with products, where each product has an ID, that is also known to the POS system. When a customer makes a purchase, the data is sent to the back-end for parsing, and is saved. Now everything is fine and dandy, until there are changes to the products name, since my client wants to see the name of the product, as it was purchased then.
How do I save this data, while also keeping a nice, normal-formed database?
Solutions I could think of are:
De-normalization, where we correlate the incoming data with the info we have in the database, and then save only the final text values, not id's.
Versioning, where we keep multiple versions of every product, and save the transactions with the id of the products version, when it came in. The problem with this one is, that as our retail store chain grows, and there are more and more changes happening to the products, the complexity of the whole product will greatly increase.
Any thoughts on this?
This is called a slowly changing dimension.
Either solution that you mention works. My preference is the second, versioning. I would have a product table that has an effdate and enddate on the record. You can easily find the current record (where enddate is null) or the record at any point in time.
The first method always strikes me as more "quick-and-dirty", but it also works. It just gets cumbersome when you have more fields and more objects you are trying to track. It does, in general though, win on performance.
If the name has to be the name as it was originally, the easiest, simplest and most reliable way to do that is to save the name of the product in the invoice line item record.
You should still link to the product with a ProductID, of course.
If you want to keep a history of name changes, you can do that in a separate table if you wish:
ProductNameID
ProductID
Date
Description
And store a ProductNameID with the invoice line item.
What is the general pattern or approach to managing transactions in a web-based e-commerce system? How do you handle the situation where more than one user tries to buy the last item, for example?
To prevent two users from purchasing the same stock item of which there is only 1 unit in stock, you need to check that each item in a user's cart has stock available right before you create an order and decrement stock for that item.
This operation will have to be atomic and only one order can be processed at any given time (read: database transaction), which should not be a problem if you are using a central database for stock management.
If stock has run out by the time a client checks out, you should remove the item from the client's cart and redirect them to their shopping cart, informing them of the situation.
Of course, this situation only occurs when two users both add the same stock item to their cart of which only one unit is in stock and one of them checks out. First come, first served. You should generally not allow clients to add products to their cart if no stock is available at that moment, unless you can order new stock within a reasonable amount of time, but in that case, the whole point is moot.
You can take a preemptive approach by checking that stock is available the moment a client initiates checkout and take the same route as above. However, that would depend on the nature of your product and the volume of transactions vs cancelled orders. If it is likely that another order for the same item was cancelled in the meantime and stock becomes available by the time a client checks out, then you don't want to lose a sale by telling the client that no stock is available. Better to let the order fail at the moment stock is not available and inform the client of the situation, which is rare after all.
Why not take the order and then get that item for the customer, perhaps a bit later? You can win a repeat customer :)