I want to open this question with the statement that i have no say in changing the datasource, which is a function in the database and is used elsewhere as well.
I have a kendo grid that i am loading from a viewmodel like this:
var data = db.Database.SqlQuery<OdometerLogsViewModel>($#"
SELECT ID, OrderNum, Carrier, Driver, Truck, StartTime, EndTime, StartOdometer, EndOdometer, Mileage, Type, Locked
FROM dbo.fnMileageLog(isnull('{StartDate}', getdate()), coalesce('{EndDate}', '{StartDate}', getdate()), {filterCarrierID}, {filterDriverID}, {filterTruckID}) ml
WHERE ('{filterType}' LIKE 'Both' OR ml.Type LIKE '{filterType}%')
AND ({id} = 0 or {id} = ml.ID)
ORDER BY Truck, StartTime DESC
this loads into the grid just fine. However i noticed when my alternate rows, which represent "dead head" of a truck and therefore have no order number, are present (not filtered out), editing a row takes a long time.
Upon inspection, it turns out that the grid is trying to "create" a new record every time i hit save for all of the null id entries. 43 of them on one page i tested. I verified this in the grid datasource requestStart event. You can not preventDefault, evidently, in that event. How do i prevent the grid from doing this? I am not opposed to hacky methods.
I got in touch with Telerik and they told me that Telerik internally does an IsNew() check on each model instance (each row) of the dataset. Since my alternating rows are not editable and are only there to identify empty miles driven, the solution was to simply modify the sql query to populate the null IDs with a fake value. We chose -1.
var data = db.Database.SqlQuery<OdometerLogsViewModel>($#"
SELECT CASE WHEN ID IS NULL THEN -1 ELSE ID END ID , OrderNum, Carrier, Driver, Truck, StartTime, EndTime, StartOdometer, EndOdometer, Mileage, Type, Locked
FROM dbo.fnMileageLog(isnull('{StartDate}', getdate()), coalesce('{EndDate}', '{StartDate}', getdate()), {filterCarrierID}, {filterDriverID}, {filterTruckID}) ml
WHERE ('{filterType}' LIKE 'Both' OR ml.Type LIKE '{filterType}%')
AND ({id} = 0 or {id} = ml.ID)
ORDER BY Truck, StartTime DESC
Related
I'm a student intern in a business team and my coworkers don't have the CS background so I hope to get some feedback and suggestion for improvement on the database design for the Flask web application that I will work on. Also, I self-learned sql a couple years ago by following tutorials on Youtube.
When a new work order is received by the business, it is then passed to a line of 5 stations to process it further. Currently the status of the work order is either started or finished. We hope to track it better by knowing the current station/stage (A, B, C, D, E) of the work order and then help improve the flow by letting the operator at each station know what's next in line.
My idea is create a web app (using Python 3, Flask, and postgresql) that updates the database when an operator at each station scans the work order's barcode and two other static barcodes (in_station_X and out_station_X). Each station will have a tablet connected to a scanner.
I. Station Operator perspective (for example Station 1)
Scan the batch of all incoming work order (barcode) for that shift. For each item, they would also scan the in_station_1 barcode to record the time_in for each work order.
The work orders come in queue so eventually the web app running on the tablet can show them what's next in line.
When an item is processed, the operator would scan the work order again and also the out_station_1 barcode to record the time_out for each work order.
The item coming out of that station may not have the same order as the incoming queue due to different priority (boolean Yes/No).
II. Admin/dashboard perspective:
See the current station and cycle time of each work order in that day.
Modify the priority of a work order if needed be.
Also, possibility to see reloop if a work order fails to be processed in station 2 and needs to go back to station 1.
III. The database:
a. Work Order Info table that contains fields such as:
id, workorder_barcode, requestor, priority (boolean Yes/No), date_created.
b. The Tracking Database: I'm thinking of having columns like:
- id (automatically generated for new row)
- workorder_barcode (nullable = False)
- current_station (nullable = False)
- time_in
- time_out
I have several questions/concerns related to this tracking table:
Every time a work order is scanned in or out, a new row will be created (which mean either column is blank). Do you see any issues with this approach vs. looking up the same work order that has time_in to fill the time_out? The reason for this is to avoid multiple look up when the database scales big.
Since the app screen at each station will show what's next in line, do you think a simple query with ORDER_BY to show the the order needed would suffice? What concerns me is showing the next item based on both Priority of each item and the current incoming order. I think I can sort by multiple columns (time_in and priority) and FILTER by current_station. However, as you can see below, I think the current table design may be more suitable for capturing events than doing queue control.
For example: the table for today would look like
id, workorder_barcode, current_station, time_in, time_out
61, 100.1, A, 6:00pm, null
62, 100.3, A, 6:01pm, null
63, 100.2, A, 6:02pm, null
...
70, 100.1, A, null, 6:03pm
71, 100.1, B, 6:04pm, null
...
74, 100.5, C, 6:05pm, null
At 6:05pm, the queue at each station would be
Station A queue: 100.3, 100.2
Station B queue: 100.1
Station C queue: 100.5
I think this can get complicated to have all 5 stations sharing the same table but seeing different queues. Is there a Queue based database that you would recommend I look into?
Thank you so much for reading this. I appreciate any questions, comments, and suggestions since I'm trying to learn more about database as I get hands-on with this project.
I've got some client code that is committing some data across some tables, in simple terms like so:
Client [Id, Balance, Timestamp]
ClientAssets [Id, AssetId, Quantity]
ClientLog [Id, ClientId, BalanceBefore, BalanceAfter]
When the customer buys an asset, I do the following pseudo code:
BEGIN TRANSACTION
GetClientRow Where ID = 1
Has enough balance for new asset cost? Yes...
Insert Into ClientAssets...
UpdateClient -> UPDATE Client SET Balance = f_SumAssetsForClient(1) WHERE ID = 1 and Timestamp = TS From Step 1;
GetClientRow Where ID = 1
Insert Into ClientLog BalanceBefore = Balance at Step 1, BalanceAfter = Balance at Step 5.
COMMIT
On step 4, the client row is updated in 1 update statement using a function 'f_SumAssetsForClient' that just sums the assets for the client and returns the balance of those assets. Also on Step 4, the timestamp is automatically updated.
My problem is, when I call GetClientRow again on Step 5, someone could have updated the clients balance, so when I go to write the log in Step 6, its not truly the balance after this set of steps. It would be the balance after a different write outside of this transaction.
If I could get the newly updated timestamp from the client row when I call UPDATE in Step 4, I could pass this to step to only grab the client row where the TS = the new updated TS. Is this possible at all? Or is my design flawed. I can't see a way out of the problem of stale data between step 5 and 6. I sense there is a problem in the table design but can't quite see it.
Step 1 needs to be SELECT ... FOR UPDATE. Any other data that needs to change also need to be "locked" FOR UPDATE.
That way, another thread cannot sneak in and modify those rows. They will probably be delayed until after you have COMMITted, or there might be a Deadlock. Either way, the thing you are worried about cannot happen. No timestamp games.
Copied from comment
Sounds like you need a step 3.5 that is SELECT f_SumAssetsForClient(1) then
store that value, then do the update, then write the log with the values - you
shouldnt have to deal with the timestamp at all -- or do the whole procedure as
a stored proc
I have no background with MySQL and have not been able to find the appropriate stepping stones to accomplish writing this particular query.
Objective:
I have created a report that shows the approval status of artwork per page. Artists have to access the proof however to determine if there is any markup/notes on each page. I want to add single column to the report that will have one of three string values per page for each proof. The string will either be "Yes", No", or "Missed" based on certain conditions.
Each proof can have multiple pages. Each page can have multiple marks (think of drawing a circle around something). Each mark can have multiple notes. This is where, for me, things get hazy. I am aware of the need to avoid RBAR queries, but I am unsure how to query against multiple comments and then marks by page.
Conditions:
This is the RBAR query I have for handling the notes/comments. This is fine for dealing with each comment, but obviously on a report there is no reason to see each comment row.
SELECT *,
CASE WHEN note IS NULL OR ' ' AND deleted = FALSE
THEN 'Missed' ELSE 'Yes'
END AS 'Comment'
FROM rvm_comment
Knowing that a mark can have multiple comments, I need to determine if any rows in rvm_comment.note are NOT NULL or contain just a space, ' '. If so, the mark is represented as "Yes". Otherwise the mark would be "Missed".
This should then be condensed/nested in a manner that each mark is compared.
If any mark on a page is "Yes" then output in that row would be "Yes". Otherwise, "Missed".
If this doesn't make sense, hopefully the following information will shed some light.
Tables and Relationships:
DSE_OBJECT Table:
This table is associated with a request that a proof can be attached to. The ID is the PK.
RVM_REVIEW_OBJECT Table:
This table is essentially the proof instance itself. A proof can have any number of pages (min 1). ID is the PK, review_object is the FK.
RVM_MARK Table:
This table contains information on marks that are added to a proof, including the page they exist on. ID is the PK, review_object is the FK. (NOT LISTED) Deleted is used to prevent returning results where a mark is deleted by the user. rvm_mark.deleted = FALSE
RVM_COMMENT Table:
This table is where the notes (strings) are stored. Deleted is used to prevent returning results where a note is deleted by the user (rvm_comment.deleted = FALSE). There is a flaw in the system where a comment can be created but if the user doesn't hit "enter" on their keyboard after typing the text is not saved in to the DB. This is why we need to test for NULL in rvm_comment.note.
Raw Data for Testing and Summary: Dropbox with CSVs for the tables
As a courtesy I have included some raw data in CSV form for anyone will to try it out. (click Dropbox link above).
So, to summarize again. I am trying to write a query that will condense those tables to a single string (AS Comment),for each rvm_mark.page_no. The string is based on a) whether or not rvm_comment.note is NULL or ' ', and b) whether any rvm_mark.id that has matching rvm_mark.page_no, has rvm_comment.note that isn't NULL.
EDIT UPDATE 12/14/16:
Thanks to Barmar I was able to take a step towards the final result. I am currently stuck on trying to return the string 'No' for any situations where there are no comments. This should only be when rvm_review_object.id does not have a matching value in rvm_mark.review_object.
SELECT rvm_review_object.dse_object_id, rvm_review_object.id,
T_Mark.creator, T_Mark.review_object,
CASE WHEN T_Mark.review_object IS NULL THEN 'No'
ELSE T_Mark.Comment
END AS Comment
FROM rvm_review_object
LEFT JOIN(
SELECT rvm_mark.review_object, rvm_mark.creator, rvm_mark.id,
CASE WHEN MAX(T_Comment.Comment = 'Yes') = 1 THEN 'Yes'
ELSE 'Missed'
END AS Comment
FROM rvm_mark
LEFT JOIN(
SELECT rvm_comment.mark,
CASE WHEN MAX((rvm_comment.note IS NULL OR rvm_comment.note = '')
AND rvm_comment.deleted = FALSE) = 1
THEN 'Missed'
ELSE 'Yes'
END AS Comment
FROM rvm_comment
GROUP BY rvm_comment.mark) AS T_Comment
ON T_Comment.mark = rvm_mark.id
WHERE rvm_mark.deleted = FALSE
GROUP BY rvm_mark.review_object) AS T_Mark
ON T_Mark.review_object = rvm_review_object.id
WHERE rvm_review_object.creator != T_Mark.creator
You need to group the query by mark. You can then use MAX() to determine if any of the rows in the group match the condition.
SELECT mark,
CASE WHEN MAX((note IS NULL OR note = '') AND deleted = FALSE) = 1
THEN 'Missed'
ELSE 'Yes'
END AS Comment
FROM rvm_comment
GROUP BY mark
OK... So I have a calendar, in a custom PHP application built using CodeIgniter, which pulls its entries from a MySQL database. In the table containing the entry data, there are 3 fields that govern the date (start_date), time (start_time) and duration (duration) of the entry. Now, as the end-user moves an entry to a new date and/or time, the system needs to check against these 3 fields for any schedule conflicts.
Following are some vars used in the query:
$request_entry_id // the id of the entry being moved
$request_start_date // the requested new date
$request_start_time // the requested new time
$request_duration // the duration of the entry being moved (will remain the same)
$end_time = ($request_start_time + $request_duration); // the new end time of the entry being moved
My query used to check for a schedule conflict is:
SELECT t.id
FROM TABLE t
WHERE t.start_date = '$request_start_date'
AND (j.start_time BETWEEN '$request_start_time' AND '$end_time'))
AND t.id <> $request_entry_id
The above query will check for any entry that starts on the same date and time as the request. However, I also want to check to make sure that the new request does not fall within the duration of an existing entry, in the most efficient way (there's the trick). Any suggestions? Thanks in advance!
It's easier to figure out the logic if you first think about the condition for when there is no conflict:
The new event ends before the existing one starts, or starts after the existing event ends.
For there to be a conflict we take the negative of the above:
The new event ends after the existing one starts, and starts before the existing event ends.
In SQL:
SELECT t.id
FROM TABLE t
WHERE t.start_date = '$request_start_date'
AND ('$end_time' > t.start_time AND '$request_start_time' < addtime(t.start_time, t.duration))
AND t.id <> $request_entry_id
I'm working on an ordering system that works exactly the way Netflix's service works (see end of this question if you're not familiar with Netflix). I have two approaches and I am unsure which approach is the right one; one relies on database polling and the other is event driven.
The following two approaches assume this simplified schema:
member(id, planId)
plan(id, moviesPerMonthLimit, moviesAtHomeLimit)
wishlist(memberId, movieId, rank, shippedOn, returnedOn)
Polling: I would run the following count queries in wishlist
Count movies shippedThisMonth (where shippedOn IS NOT NULL #memberId)
Count moviesAtHome (where shippedOn IS NOT NULL, and returnedOn IS NULL #memberId)
Count moviesInList (#memberId)
The following function will determine how many movies to ship:
moviesToShip = Min(moviesPerMonthLimit - shippedThisMonth, moviesAtHomeLimit - moviesAtHome, moviesInList)
I will loop through each member, run the counts, and loop through their list as many times as moviesToShip. Seems like a pain in the neck, but it works.
Event Driven: This approach involves adding an extra column "queuedForShipping" and marking it to 0,1 every time an event takes place. I will do the following counts:
Count movies shippedThisMonth (where shippedOn IS NOT NULL #memberId)
Count moviesAtHome (where shippedOn IS NOT NULL, and returnedOn IS NULL #memberId)
Count moviesQueuedForShipping (where queuedForShipping = 1, #memberId)
Instead of using min, I have to use the following if statements
If moviesPerMonthLimit > (shippedThisMonth + moviesQueuedForShipping)
AND IF moviesAtHomeLimit > (moviesAtHome + moviesQueuedForShipping))
If both conditions are true, I will select a row from wishlist where queuedForShippinh = 0, and set it's queuedForShipping to 1. I will run this function every time someone adds, deletes, reorders their list. When it's time to ship, I would select #memberId where queuedForShipping = 1. I would also run this when updating shippedAt and returnedAt.
Approach one is simple. It also allows members to mess around with their ranks until someone decides to run the polling. That way what to ship is always decided by rank. But ppl keep telling polling is bad.
The event driven approach is self-sustaining, but it seems like a waste of time to ping the database with all those counts every time a person changes their list. I would also have to write to the column queuedForShipment. It also means when a member re-ranks their list and they have pending shipments (shippedAt IS NULL, queuedForShipping = 1) I would have to update those rows and set queuedForShipping back to 1 based on the new ranks. (What if someone added 5 movies, and then suddenly went to change the order? Well, queuedForShipment would already be set to 1 on the first two movies he or she added)
Can someone please give me their opinion on the best approach here and the cons/advantages of polling versus event driven?
Netflix is a monthly subscription service where you create a movie list, and your movies are shipped to you based on your service plan limits.
Based on what you described, there's no reason to keep the data "ready to use" (event) when you can create it very easily when needed (poll).
Reasons to cache it:
If you needed to display the next item to the user.
If the detailed data was being removed due to some retention policy.
If the polling queries were too slow.