This question is related to the following question:
How do I INSERT INTO t1 (SELECT * FROM t2) in SQLAlchemy?
but I'd like to specify which columns the insert should work on. That is, I'd like sqlalchemy to generate a query equivalent to
INSERT INTO t1 (col1, col2, col3) SELECT x,y,z FROM t2
I looked at the compilation documentation, but it's not clear to me how to modify the example to be able to specify the column names.
Following modifications might help:
from sqlalchemy.ext.compiler import compiles
from sqlalchemy.sql.expression import Executable, ClauseElement
class InsertFromSelect(Executable, ClauseElement):
def __init__(self, table, columns, select):
self.table = table
self.columns = columns
self.select = select
#compiles(InsertFromSelect)
def visit_insert_from_select(element, compiler, **kw):
return "INSERT INTO %s (%s) %s" % (
compiler.process(element.table, asfrom=True),
", ".join(element.columns), # #note: not a very safe/robust way to compose SQL
compiler.process(element.select)
)
insert = InsertFromSelect(
t1,
("col1", "col2", "col3",),
select([t2.c.x, t2.c.y, t2.c.z])
)
print insert
Related
I don't know how to add the values by index help me with this: Add the values according to the index of the values of the Key
The input CSV:
Country,Values
Canada,47;97;33;94;6
Canada,59;98;24;83;3
Canada,77;63;93;86;62
China,86;71;72;23;27
China,74;69;72;93;7
China,58;99;90;93;41
England,40;13;85;75;90
England,39;13;33;29;14
England,99;88;57;69;49
Germany,67;93;90;57;3
Germany,0;9;15;20;19
Germany,77;64;46;95;48
India,90;49;91;14;70
India,70;83;38;27;16
India,86;21;19;59;4
The output csv Should be:
Country,Values
Canada,183;258;150;263;71
China,218;239;234;209;75
England,178;114;175;173;153
Germany,144;166;151;172;70
India,246;153;148;100;90
Import required modules & create session
from pyspark.sql import SparkSession
spark_session = SparkSession.builder.getOrCreate()
Read csv file & create view
file_df = spark_session.read.csv("/data/user/country_values.csv",header=True)
temp_view = file_df.createOrReplaceTempView("temp_view")
Write sql query
output_df = spark_session.sql("select country, concat( cast (sum (split(values,';') [0]) as int),';', cast (sum (split(Values,';')[1]) as int),';',cast(sum(split(values,';')[2]) as int),';', cast (sum (split(Values,';')[3]) as int),';', cast(sum (split(values,';')[4]) as int)) as values from temp_view group by Country order by country")
print result
output_df.show()
Limitations -> SQL query has been written in a way that Values column should have 5 values after splitting with ";" delimiter. Values column is being split 5 times in query, it can be optimized.
Or use CTE to split the Values into an array of integers before the main aggregation:
spark_session.sql("""
WITH t1 AS (
SELECT Country
, CAST(split(Values, ";") AS array<int>) AS V
FROM temp_view
)
SELECT Country
, CONCAT_WS(";", sum(V[0]), sum(V[1]), sum(V[2]), sum(V[3]), sum(V[4])) AS Values
FROM t1
GROUP BY Country
ORDER BY Country
""").show(truncate=False)
I'm using the mysql-otp driver for Erlang. It seems to be working fine but there is no documentation on using it to insert multiple rows into a table.
simple use case for single row insert:
ok = mysql:query(Pid, "INSERT INTO mytable (id, bar) VALUES (?, ?)", [1, 42]).
But I need to insert multiple values, can I do something like this?
ok = mysql:query(Pid, "INSERT INTO mytable (id, bar) VALUES (?, ?)", [(1, 42),(2, 36), (3,12)]).
Documentation states Params = [term()], so probably not, which is a bummer.
You can certainly do a combination of lists:foldl/3 and lists:join/2 on your arguments to create the desired query format:
L = [[1, 42],[2, 36], [3,12]],
PreparedList = lists:foldl(fun (Params, Inserts) -> Inserts ++ [io_lib:format("(~p,~p)", Params)] end, [], L),
%% Then you need to join these with a comma:
Prepared = lists:flatten(lists:join(",", PreparedList)),
%% this will result in "(1,42),(2,36),(3,12)"
Now you just need to call the mysql insert with this Prepared variable:
ok = mysql:query(Pid, "INSERT INTO mytable (id, bar) VALUES ?", [Prepared]).
%% The query will look like: "INSERT INTO mytable (id, bar) VALUES (1,42),(2,36),(3,12)"
I don't think this driver or mysql can do such kind of things.
I think you should do it likes below
insert_mytable(Data)->
{ok,Ref} = mysql:prepare(Pid,insert_mytable,"INSERT INTO mytable (id, bar) VALUES (?, ?)"),
loop_insert(_Pid,Ref,Data).
loop_insert(_Pid,_Ref,[])-> ok;
loop_insert(Pid,Ref,[H|T])->
ok = mysql:execute(Pid,Ref,H),
loop_insert(Pid,Ref,T).
How can I generate this query using SQLAlchemy's ORM layer?
insert into foo (a,b)
select ‘new’ as a, b
from foo
where a = ‘old’;
Assume I have foo already mapped to a class.
I have seen Insert.from_select, but I'm not sure how I can modify one of the fields like I'm doing in the text query.
A combination of select(), where() and insert().from_select() can do the job:
>>> from sqlalchemy.sql import table, column, select
>>> foo = table('foo', column('a'), column('b'))
>>> print(foo.insert().from_select(['a', 'b'],
select(['"new" as a', 'b']).select_from(foo).where('a = "old"')))
INSERT INTO foo (a, b) SELECT "new" as a, b
FROM foo
WHERE a = "old"
Hope that helps.
I have a table with columns machine id like (311a__) and (311bb__) and some of them like (08576). How can I retrieve them in SQL - how to insert into row where machine ID is like ( 311a__)? My question how to insert and select a column which has spaces in it.. How to retrieve data where machine_name ="%s__" is it correct
sql_local = """SELECT id FROM customer_1.pay_machines WHERE machine_name="%s" """ % machine
retVal = cursor.execute(sql_local)
if (retVal == 0):
sql_local = """INSERT INTO customer_1.pay_machines (machine_name, carpark_id) VALUES ("%s", 0)""" % machine
Surround odd (or reserved word) column names with backticks:
SELECT *
FROM pd
WHERE `machine id` = '(%s__)';
edit: removed invalid insert query as the first query is sufficient as an example
Can someone tell me why this insert is failing but not giving me an error either? How do I fix this?
merge table1 as T1
using(select p.1,p.2,p.3,p.4,p.5 from #parameters p
inner join table1 t2
on p.1 = t2.1
and p.2 = t2.2
and p.3 = t2.3
and p.4 = t2.4) as SRC on SRC.2 = T1.2
when not matched then insert (p.1,p.2,p.3,p.4,p.5)
values (SRC.1,SRC.2,SRC.3,SRC.4,SRC.5)
when matched then update set t1.5 = SRC.5;
The T1 table is currently empty so nothing can match. The parameters table does have data in it. I simply need to modify this merge so that it checks all 4 fields before deciding what to do.
You can't select from a variable: from #parameters
See the following post: Using a variable for table name in 'From' clause in SQL Server 2008
Actually, you can use a variable table. Check it out:
MERGE Target_table AS [Target]
USING #parameters AS [Source]
ON (
[Target].col1 = [Source].col1
AND [Target].col2 = [Source].col2
AND [Target].col3 = [Source].col3
AND [Target].col4 = [Source].col4
)
WHEN NOT MATCHED BY TARGET
THEN INSERT (col1,col2,col3,col4,col5)
VALUES (
[Source].col1
,[Source].col2
,[Source].col3
,[Source].col4
,[Source].col5
)
WHEN MATCHED
THEN UPDATE SET [Target].col5 = [Source].col5;