I am doing a migration where I have a table A that I'm splitting into table B and C. My (simplified) migration script is :
from alembic import op
con = op.get_bind()
items = con.execute("SELECT * FROM public.A")
for item in items:
B_id = item.b
C_id = item.c
some_attributes = item.some_attributes
some_other_attributes = item.some_other_attributes
created_at = item.created_at
updated_at = created_at
if item.deleted:
deleted_at = created_at
else:
deleted_at = created_at # This should be None
op.execute("INSERT INTO B (id, some_attributes, created_at, updated_at, deleted_at)\
VALUES ('{}','{}','{}','{}','{}')\
ON CONFLICT DO NOTHING".format(B_id, some_attributes, created_at, updated_at, deleted_at))
op.execute("INSERT INTO C (id, some_other_attributes ,created_at, updated_at, deleted_at, B_id)\
VALUES ('{}','{}','{}','{}','{}','{}')".format(C_id,some_other_attributes,created_at,updated_at,deleted_at,B_id))
For now this works but the thing is that when my item is not deleted I would like my deleted_at to be set to NULL. The thing is that if I set the value to None, "NULL", "null" or even sqlalchemy.sql.null() I get this error :
sqlalchemy.exc.DataError: (psycopg2.errors.InvalidDatetimeFormat) invalid input syntax for type timestamp: "NULL"
My column is nullable, I can do what I want by duplicating my code in the if item.deleted statement and not mention the deleted _at column when I want to set it to null but that's not very elegant at all.
I could also import A, B and C from my models and create a new instance of B and C each time and add then commit each one to my session, but this would require me to change B and C init code and that feels wrong to me to do so just for migration sake.
Does anyone have any idea for me ? Tell me if I need to edit anything to add precisions.
I finally managed to do it with this approach :
I could also import A, B and C from my models and create a new instance of B and C each time and add then commit each one to my session, but this would require me to change B and C init code and that feels wrong to me to do so just for migration sake.
But instead of changing the init code for B and C i just did this :
b = B(some_attributes,created_at)
b.deleted_at = deleted_at
session.add(b)
session.commit()
Related
I have two tables.
Table number 1 (Lab_test): with integer columns named labtest_id (pk) and project_no. Second table (Project) has two columns, first column named project_id (int) and second column project_name (string). Also I have two parameters which are passed to the query.
I have to make a query which will extract labtest_id and proj_num. proj_num have to be obtained in following way: if Lab_test.project_no is not null, take this value, else go to referencing table Project and take project_name.
I tried with bunch of combinations using SQL CASE IF-THEN-ELSE statements, but no luck:
SELECT lab.labtest_id, proj_num,
FROM Lab_test lab
(CASE
WHEN lab.project_no IS NOT NULL THEN lab.project_no
ELSE (SELECT proj.project_name
FROM Project proj
WHERE proj.project_id = lab.project_no)
) AS proj_num
WHERE lab.status = 'DONE' AND lab.user_id = 436
You can use left join and coalesce() to replace the case when expression that you've used to check null
SELECT lab.labtest_id, coalesce(lab.project_no,proj.project_name) as proj_num
FROM Lab_test lab
left join Project proj on proj.p,roject_id = lab.project_no
where lab.status = 'DONE' AND lab.user_id = 436
if Lab_test.project_no is not null, take this value, else go to referencing table Project and take project_name.
I don't believe you. I think you have the logic backwards -- use the referenced table if it is available. Otherwise use the number.
That would suggest:
SELECT l.labtest_id, COALESCE(p.project_name, l.project_no) as proj_num
FROM Lab_test l LEFT JOIN
Project p
ON p.p,roject_id = l.project_no
WHERE l.status = 'DONE' AND l.user_id = 436;
If the project_no is a number, then you might have a type conversion problem. If that is an issue:
SELECT l.labtest_id,
COALESCE(p.project_name, CAST(l.project_no as CHAR)) as proj_num
FROM Lab_test l LEFT JOIN
Project p
ON p.p,roject_id = l.project_no
WHERE l.status = 'DONE' AND l.user_id = 436;
I have a database tables of transactions. Because of some flukes in the code, there have been a number of cases where positive transactions, and negative transactions have been made on the same invoice, in the same amount. There is a soft delete column named void. I am trying to update this, to void matching pairs.
Basically what I want is this:
UPDATE transactions pos JOIN transactions neg on pos.invoice_id = neg.invoice_id
AND pos.amount = neg.amount AND pos.effect = 1 AND neg.effect = -1
AND pos.void IS NULL AND neg.void IS NULL
SET pos.void = true, neg.void = true
Just running this as-is though, will not uniquely identify pairs. So, if there is a +X, -X, +X, or alternatively -Y, +Y, -Y for an invoice, it would void all. I would have thought this would be a common problem, but I am not seeing any questions or answers that quite fit my problem.
Most answers to similar questions involve using a sub-query, but they are only trying to update 1 entry, not 2, and I don't see how I'd be able to update both entries when doing that.
Thanks,
-Eric
maybe if you post your table structure i can help you better, but i think this is what you want:
UPDATE transactions
SET void = true
WHERE invoice_id in (
SELECT invoice_id from transactions as pos
JOIN transactions neg
ON pos.invoice_id = neg.invoice_id
AND pos.amount = neg.amount AND pos.effect = 1 AND neg.effect = -1
AND pos.void IS NULL AND neg.void IS NULL
)
I am trying to write a query where two different UIDs need to lookup a Resource Name for both, but separately.
In other words, for each Task, there are resources assigned and one status manager. This converts in SQl to an Assignment, unique to a resource, but with the same status manager. However, no where in the database can one see the Status Manager's Name on a given assignment.
The assignment does have "TaskStatusManagerUID" available. The name of the Status Manager can be determined by tying it back to MSP_EPMResource table where TaskStatusManagerUID = ResourceUID.
The catch is, for my report, I need to be able to look at the ResourceUID and TaskstatusManagerUID and determine the names of each on the same assignment.
While I have been successful with a join to display the name for one or the other, I have not been able to determine how to show the name for both the Resource and TaskStatusManager.
This is an example of what I am trying to display (parentheses added for readability):
(AssignmentUID) (Task Name) (Resource Name) (Task Status Manager Name)
See more info below:
This is the code I have been working with, but have been unsuccessful:
Select top 100
c.[assignmentuid],
a.[taskname],
c.[resourceuid],
b.[resourcename],
a.[taskstatusmanageruid],
d.[StatusManager]
from [PRJPROD_ProjectWebApp].[dbo].[MSP_EpmAssignment] c
join [PRJPROD_ProjectWebApp].[dbo].[MSP_EpmTask_UserView] a
on a.[TaskUID] = c.[TaskUID]
join [PRJPROD_ProjectWebApp].[dbo].[MSP_EpmResource] b
on b.[ResourceUID] = c.[ResourceUID]
join (select b.resourcename StatusManager
from [PRJPROD_ProjectWebApp].[dbo].[MSP_EpmResource] b) d
on d.[StatusManager] = a.[taskstatusmanageruid]
group by
c.[assignmentuid],
a.[taskname],
c.[resourceuid],
b.[resourcename],
a.[taskstatusmanageruid],
d.[StatusManager]
Currently, I am getting "Conversion failed when converting from a character string to uniqueidentifier."
On your joins you have on a.[TaskUID] = c.[TaskUID], on b.[ResourceUID] = c.[ResourceUID], and on d.[StatusManager] = a.[taskstatusmanageruid], of which, I am assuming that the last one is causing you the issue. Try instead
join (select b.resourcename StatusManager
from [PRJPROD_ProjectWebApp].[dbo].[MSP_EpmResource] b) d
on d.[StatusManager] = CONVERT(CHAR, a.[taskstatusmanageruid])
This will convert the GUID contained in taskstatusmanageruid to a char string, allowing it to compare successfully.
You could also, instead of converting the value, cast the value CAST(a.[taskstatusmanageruid] AS CHAR
EDIT
Due to the nature of the GUID, you may not be able to convert/cast it to a char value, in which case you would need to convert/cast both fields to either varchar or nvarchar:
join (select b.resourcename StatusManager
from [PRJPROD_ProjectWebApp].[dbo].[MSP_EpmResource] b) d
on CONVERT([N]VARCHAR, d.[StatusManager]) = CONVERT([N]VARCHAR, a.[taskstatusmanageruid])
OR
join (select b.resourcename StatusManager
from [PRJPROD_ProjectWebApp].[dbo].[MSP_EpmResource] b) d
on CAST(d.[StatusManager] AS [N]VARCHAR) = CAST( a.[taskstatusmanageruid] AS [N]VARCHAR)
Thanks to Jeff Beese's extra set of eyes, it was enough for me to get the last piece in place!
Select top 100
c.[assignmentuid],
a.[taskname],
c.[resourceuid],
b.[resourcename],
a.[taskstatusmanageruid],
d.[StatusManager]
from [PRJPROD_ProjectWebApp].[dbo].[MSP_EpmAssignment] c
join [PRJPROD_ProjectWebApp].[dbo].[MSP_EpmTask_UserView] a
on a.[TaskUID] = c.[TaskUID]
join [PRJPROD_ProjectWebApp].[dbo].[MSP_EpmResource] b
on b.[ResourceUID] = c.[ResourceUID]
join (select b.resourcename as StatusManager,
b.ResourceUID
from [PRJPROD_ProjectWebApp].[dbo].[MSP_EpmResource] b) d
on d.[resourceuid] = a.[taskstatusmanageruid]
group by
c.[assignmentuid],
a.[taskname],
c.[resourceuid],
b.[resourcename],
a.[taskstatusmanageruid],
d.[StatusManager]
Here is the code from django docs that explains the use of managers.
class PollManager(models.Manager):
def with_counts(self):
from django.db import connection
cursor = connection.cursor()
cursor.execute("""
SELECT p.id, p.question, p.poll_date, COUNT(*)
FROM polls_opinionpoll p, polls_response r
WHERE p.id = r.poll_id
GROUP BY p.id, p.question, p.poll_date
ORDER BY p.poll_date DESC""")
result_list = []
for row in cursor.fetchall():
p = self.model(id=row[0], question=row[1], poll_date=row[2])
p.num_responses = row[3]
result_list.append(p)
return result_list
class OpinionPoll(models.Model):
question = models.CharField(max_length=200)
poll_date = models.DateField()
objects = PollManager()
class Response(models.Model):
poll = models.ForeignKey(OpinionPoll)
person_name = models.CharField(max_length=50)
response = models.TextField()
I have two questions based on this code:
1) where is r.poll_id coming from? I understand Response has foreignKey relationship to OpinionPoll. In order to JOIN OpinionPoll table with Response table, I need to join on their id.
HOwever to access the poll id in Response, I would do r.poll.id.
Is the syntax, r.poll_id, a MySQL syntax.
why GROUP BY p.id, p.question, p.poll_date? why GROUP BY p.id alone is not sufficient?
2) Is it possible to turn the above raw SQL query into a django ORM query?If so how would that look like?
I am not a SQL guy. so bear with me, if this sounds stupid
EDIT:
If I want to create OpinionPoll and Response tables outside of Django, how will SQL statment for create look like?
In the Django shell, when I run
python manage.py sqlall appname
I get the following:
BEGIN;
CREATE TABLE "myapp_opinionpoll" (
"id" integer NOT NULL PRIMARY KEY,
"question" varchar(200) NOT NULL,
"poll_date" date NOT NULL
)
;
CREATE TABLE "myapp_response" (
"id" integer NOT NULL PRIMARY KEY,
"poll_id" integer NOT NULL REFERENCES "myapp_opinionpoll" ("id"),
"person_name" varchar(50) NOT NULL,
"response" text NOT NULL
)
;
CREATE INDEX "larb_response_70f78e6b" ON "myapp_response" ("poll_id");
COMMIT;
I see something like REFERENCES "myapp_opinionpoll" and CREATE INDEXabove. I am not sure
if this is how in SQL it is done?
[1] Django model will create foreign keys like fieldname_id as the field in mysql. So you see the field poll = models.ForeignKey(OpinionPoll) creates this field.
About GROUP BY, because these fields are exactly what selected, except for the aggregate function, grouping them exactly can make them distinct.
[2] Try this, I didn't debug, but may helps:
from django.db.models import Count
OpinionPoll.objects.annotate(num_responses=Count('response'))
For more about aggregation, see the docs: https://docs.djangoproject.com/en/1.6/topics/db/aggregation/
First of all I'm rather new to SQL and so even though I believe a similar question was asked in this thread ( SQL Query - Copy Values in Same Table ) I literally can't understand it well enough to utilize the information. For that I apologize.
Now, I have a table that looks something like this:
company id | parameter name | parameter title
P | Parameter One | First Parameter
P | Parameter Two | Second Parameter
P | Parameter Three| Third Parameter
W | Parameter One | NULL
W | Parameter Two | NULL
Except that my table obviously has quite a lot of rows. I already went through filling in all the parameter titles where the company id was 'P' and would like to avoid manually doing the same for those with company id 'W'. My question is what SQL statement (this is in Microsoft SQL Server 2008) can I use to copy the values in the column "parameter title" where the company id is 'P' to the values in the same column where the company id is 'W' and both parameter names match up (W has less parameters than P)?
Using the previously linked thread I was able to come up with the following, but it spits out an error and I know it's not done correctly:
UPDATE COMP_PARAMETER_COPY
SET PARAM_TITLE=(SELECT PARAM_TITLE FROM COMP_PARAMETER_COPY P
WHERE P.COMP_ID = 'P' AND P.PARAM_TITLE=PARAM_TITLE)
WHERE COMP_ID='W'
(I'm playing around with a copy of the table instead of the actual table)
The error I get is "Msg 512, Level 16, State 1, Line 1
Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression.
The statement has been terminated."
Thank you for your help and advice,
-Asaf
You need to ensure that your subquery is only returning one result. Right now that error message is telling you that you're getting more than one record returned.
UPDATE W
SET PARAM_TITLE = (
SELECT PARAM_TITLE FROM COMP_PARAMETER_COPY P
WHERE P.COMP_ID = 'P' AND P.PARAM_NAME = W.PARAM_NAME
)
FROM COMP_PARAMETER_COPY W
WHERE W.COMP_ID = 'W'
Try giving the above SQL a whirl. This could still give you more than one result, but without knowing what your table looks like and what the data constraints are it's hard to give you something guaranteed to work.
Try adding the DISTINCT keyword to your query:
UPDATE COMP_PARAMETER_COPY
SET PARAM_TITLE=(SELECT DISTINCT PARAM_TITLE FROM COMP_PARAMETER_COPY P
WHERE P.COMP_ID = 'P' AND P.PARAM_TITLE=PARAM_TITLE)
WHERE COMP_ID='W'