User may import a excel and I want to check if the data are correct.
# Excel Data
| id | item | qty |
|:---- |:------:| -----:|
| 1 | item A | 10 |
| 2 | item B | 20 |
| 3 | item C | 30 |
| 4 | item D | 40 | <-- For example, Not enough qty to minus (only have 1)
| 5 | item E | 50 |
# Database
| id | item | qty |
|:---- |:------:| -----:|
| 1 | item A | 100 |
| 2 | item B | 200 |
| 3 | item C | 300 |
| 4 | item D | 1 | <-- For example, Not enough qty to minus (Need 40)
| 5 | item E | 500 |
I need to check the Database is that item has qty to minus, if yes then save the changes, if not, then rollback all changed data in this excel data (rollback to before import this excel) and return errors details to user.
def myFunction(self, request):
try:
error_details = []
with transaction.atomic():
for data in excal_data:
result = checkIfVerify(data) # Here will be a function which will cause error 'You can't execute queries until the end of the 'atomic' block'
if result is True:
serializer = modelSerailizer(data)
serializer.save()
else:
error_details.append("some explanation...")
if len(error_details) > 0:
transaction.set_rollback(True)
raise CustomError
excpet CustomError:
pass
return Response(....)
# checkIfVerify(data)
def checkIfVerify(data):
# this sql will need to join many tables which is hard to use ORM
sql = ....
results = []
with connection.cursor() as cursor:
cursor.execute(sql)
results = cursor.fetchall()
cursor.close()
connection.close()
if results .....:
return True
else:
return False
But the problem seem to be I cannot able to use raw SQL execute inside the transaction.atomic() block, If I put transaction.atomic() inside the loop after the checking function, it not able to rollback all data.
How should I do?
Related
I am trying to create a query in ms access which sums up costs but want to exclude those that have no value from the result. I am struggling because I want to exclude items from one column based on another column.
I have a column with finished product, column with components that make up finished product, column with quantity of components required for finished product and column with cost for each of components. What I need is to get total cost for each component required and then sum up costs as total cost for finished product which is simple enough.
However there are some blank fields where cost for one or more components are not available, I want those to show in the result with text " Pending " for finished product instead of it summing up only available values.
Below example of what I am trying to do:
BOM_List
What I need in result is below:
BOM_Result
Would really appreciate any help on this :)
Elaborating on June 7ths comment and assuming Table2:
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| ID | FinishedProductid | ItemNumber | Component | Uom | ComponentQuantity | ComponentUnit | stdCost |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 1 | 199127402 | 10 | 123123123 | PC | 3 | PC | $1.50 |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 2 | 199127402 | 20 | 321321321 | PC | 1 | PC | $2.50 |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 3 | 199127402 | 30 | 123456789 | PC | 1 | PC | $3.55 |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 4 | 199127402 | 40 | 987654321 | PC | 0.25 | H | $82.00 |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 5 | 199127403 | 10 | 111222333 | PC | 3 | PC | $1.50 |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 6 | 199127403 | 20 | 333222111 | PC | 1 | PC | |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 7 | 199127403 | 30 | 444555666 | PC | 1 | PC | $3.55 |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 8 | 199127403 | 40 | 666555444 | PC | 0.25 | H | $82.00 |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
'resulting sql to get all FinishedProductids
SELECT Table2.FinishedProductid
FROM Table2
GROUP BY Table2.FinishedProductid;
'once have FinishedProductid's out of the group statement you can just use a calculated variable
SELECT Query2.FinishedProductid, getTotalCost([FinishedProductid]) AS TotalCost
FROM Query2;
'of course a getTotalCost function is needed. Add the following to a code module
Public Function getTotalCost(FinishedProductid As Long) As String
If isPending(FinishedProductid) Then
getTotalCost = "Pending"
Else
getTotalCost = DSum("ComponentQuantity*stdCost", "Table2", "FinishedProductid = " & FinishedProductid)
End If
End Function
Public Function isPending(FinishedProductid) As Boolean
' if any values of stdCost are null set isPending to true
'public functions can be accessed most anywhere in Access
Dim nullCount As Long
nullCount = DCount("ID", "Table2", "FinishedProductid = " & FinishedProductid & " AND ISNULL(stdCost)")
If nullCount > 0 Then
isPending = True
Else
isPending = False
End If
End Function
'result
-------------------------------------------------------
| FinishedProductid | TotalCost |
-------------------------------------------------------
| 199127402 | 31.05 |
-------------------------------------------------------
| 199127403 | Pending |
-------------------------------------------------------
Explanation: I use a two query approach to get around the sql complications that arise from the group by. The functions are relatively declaritive and hence self-commenting. Finally, functions can be reused.
Edit: getTotalCost returns a string so it can meet the requirement of returning both the string "pending" and the total cost
I have a table queue and I want to know what position or row number a user at.
queue table
----------------------------------------------------
| id | name | created_at | done |
+-----+--------+-------------------------+---------+
| 1 | John | 2020-10-17 01:08:59 | 1 |
| 2 | Jane | 2020-10-17 01:10:15 | 0 |
| 3 | Jess | 2020-10-17 01:18:15 | 0 |
| 4 | Joe | 2020-10-18 08:18:15 | 0 |
| 5 | Moe | 2020-10-18 11:18:15 | 0 |
----------------------------------------------------
is it possible to know the specific number of user in queue? for example Jess will return 3 because he's the 3rd user in the queue record.
edit: for example John is done in the queue, now Jess will become the 2nd in the queue.
I think I have a way:
the main idea is to get the count of the previous queue based on their id values,
and the trick is to use an alias to the main table so you can use that alias in your internal select.
$values = DB::table('queue', 'u1')
->select('u1.id',DB::raw("((SELECT count(*) from queue WHERE queue.id < u1.id)+1) rowNumber"))
->orderBy('u1.id')
->get();
Edit:
if you want to exclude done queue you should do it in the main and the internal select:
$values = DB::table('queue', 'u1')
->select('u1.id',DB::raw("((SELECT count(*) from queue WHERE (queue.id < u1.id)and(queue.is_done!=1) )+1) rowNumber"))
->where('u1.is_done','!=',1)
->orderBy('u1.id')
->get();
If you don't need to get that number directly with query you can do
search()
$collection->search(function ($item, $key) {
return $item->name == 'Jess';
});
I'm trying to write a query in PySpark that will get the correct value from an array.
For example, I have dataframe called df with three columns, 'companyId', 'companySize' and 'weightingRange'. The 'companySize' column is just the number of employees. The column 'weightingRange' is an array with the following in it
[ {"minimum":0, "maximum":100, "weight":123},
{"minimum":101, "maximum":200, "weight":456},
{"minimum":201, "maximum":500, "weight":789}
]
so the dataframe looks like this (weightingRange is as above, its truncated in the below example for clearer formating)
+-----------+-------------+------------------------+--+
| companyId | companySize | weightingRange | |
+-----------+-------------+------------------------+--+
| ABC1 | 150 | [{"maximum":100, etc}] | |
| ABC2 | 50 | [{"maximum":100, etc}] | |
+-----------+-------------+------------------------+--+
So for a entry for company size = 150 I need to return the weight 456 into a column called 'companyWeighting'
So it should show the following
+-----------+-------------+------------------------+------------------+
| companyId | companySize | weightingRange | companyWeighting |
+-----------+-------------+------------------------+------------------+
| ABC1 | 150 | [{"maximum":100, etc}] | 456 |
| ABC2 | 50 | [{"maximum":100, etc}] | 123 |
+-----------+-------------+------------------------+------------------+
I've had a look at
df.withColumn("tmp",explode(col("weightingRange"))).select("tmp.*")
and then joining but trying to apply that would Cartesian the data.
Suggestions appreciated!
You can approach like this,
First creating a sample dataframe,
import pyspark.sql.functions as F
df = spark.createDataFrame([
('ABC1', 150, [ {"min":0, "max":100, "weight":123},
{"min":101, "max":200, "weight":456},
{"min":201, "max":500, "weight":789}]),
('ABC2', 50, [ {"min":0, "max":100, "weight":123},
{"min":101, "max":200, "weight":456},
{"min":201, "max":500, "weight":789}])],
['companyId' , 'companySize', 'weightingRange'])
Then, creating a udf function and applying it on each row to get the new column,
def get_weight(wt,wt_rnge):
for _d in wt_rnge:
if _d['min'] <= wt <= _d['max']:
return _d['weight']
get_weight_udf = F.udf(lambda x,y: get_weight(x,y))
df = df.withColumn('companyWeighting', get_weight_udf(F.col('companySize'), F.col('weightingRange')))
df.show()
You get the output as,
+---------+-----------+--------------------+----------------+
|companyId|companySize| weightingRange|companyWeighting|
+---------+-----------+--------------------+----------------+
| ABC1| 150|[Map(weight -> 12...| 456|
| ABC2| 50|[Map(weight -> 12...| 123|
+---------+-----------+--------------------+----------------+
I have the following table of data from an MDX query that resembles the following:
Account | Location | Type | Person | Amount
ABC | XYZ | AA | Tom | 10
ABC | XYZ | AA | Dick | 20
ABC | XYZ | AA | Harry | 30
ABC | XYZ | BB | Jane | 50
ABC | XYZ | BB | Ash | 100
DEF | SDQ | ZA | Bob | 20
DEF | SDQ | ZA | Kate | 10
DEF | LAO | PA | Olivia | 200
DEF | LAO | PA | Hugh | 120
And I need to add the Amount column for each Account, Location, and Type. If I was using SQL I would perform a query on this data as follows:
Select Account, Location, Type, Sum(Amount) as SumAmmount
From Table
Group By Account, Location, Type
but due to the way we store the data I need to roll-up this data using SSRS. To do that I created a tablix, created a parent group (Which I have called "RollUp") of the default detail group which grouped on Account, Location, and Type and then deleted the detail group so when running the report I get:
Account | Location | Type | Amount
ABC | XYZ | AA | 60
ABC | XYZ | BB | 150
DEF | SDQ | ZA | 30
DEF | LAO | PA | 320
What I need to do now is create a page break so that when I export this SSRS report to excel there are only 1000 rows on each sheet, but I am having trouble writing the expression to split this every 1000 rows. Because I have removed the details group I cannot use the typical expression I would use to page break on a specific row of a dataset (e.g. Group Expression = Floor(RowNumber(NOTHING) / 1000) )
I have tried a few different things like writing some custom code and some running value expressions but haven't been able to figure it out yet.
I did figure out how to do this.
First I created the following custom code in the report definition:
Dim GroupingDummy = "GroupDummy"
Dim RowNumberToReturn = -1
Function PopulateRowNumber(GroupString As String) As Integer
If (GroupString <> GroupingDummy ) Then
GroupingDummy = GroupString
RowNumberToReturn = RowNumberToReturn + 1
End If
Return RowNumberToReturn
End Function
Keeping in mind the grouping I applied to the dataset used the fields Account, Location, and Type, I added a calculated field to my dataset with the name RowNumberCalc and the expression:
=Code.RowNumberToReturn(Fields!Account.Value + Fields!Location.Value + Fields!Type.Value)
Now I could easily create the group that would create a page break at 1000 rows with the expression :
=Floor(Fields!RowNumberCalc.Value / 1000)
I've written a query on a table which contains a list of 'Statements' against 'Companies' like so (obviously simplified):
Statement | Company
---------------------------
ABC | CompA
ABC | CompB
DEF | CompC
The query presents the information like so:
Statement | CompA | CompB | Comp C
--------------------------------------
ABC | X | X |
DEF | | | X
Using the code like this:
SELECT
[Requirement_Text],
CASE WHEN(SUM(CASE WHEN Company = 'CompA' THEN 1 END)) IS NOT NULL THEN 'X' ELSE ' ' END AS CompA,
CASE WHEN(SUM(CASE WHEN Company = 'CompB' THEN 1 END)) IS NOT NULL THEN 'X' ELSE ' ' END AS CompB,
CASE WHEN(SUM(CASE WHEN Company = 'CompC' THEN 1 END)) IS NOT NULL THEN 'X' ELSE ' ' END AS CompC,
CASE WHEN(SUM(CASE WHEN Company IS NULL THEN 1 END)) IS NOT NULL THEN 'X' ELSE ' ' END AS NILL
FROM [StatementTable]
Now, this is easy enough because we have a finite number of companies, but if we were to say move it down to Department level (instead of Company) then we have several more (an unknown number - N).
So the question is, how can I create the columns in the output table based on the number of distinct values in a given column of the input table?
For example:
Statement | Company | Department
---------------------------------------
ABC | CompA | Dept(1)
DEF | CompA | Dept(2)
DEF | CompA | Dept(3)
GHI | CompA | Dept(3)
ABC | CompB | Dept(N-1)
DEF | CompC | Dept(N)
Will become:
Statement | Dept(1) | Dept(2) | Dept(3) | Dept(N-1) | Dept(N)
---------------------------------------------------------------
ABC | X | | | X |
DEF | | X | X | | X
GHI | | | X | |
NOTE: In this case I've disregarded the company name.
Thanks in advance.
you should use Dynamic query and Pivote :
Check This out
and you can see the document here :
Check this