Trying to configure an Insert into command for MySQL - mysql

I'm pretty new to SQL and need some help configuring a command. The details of my database structure can be found in this thread:
How to copy new data but skip old data from 2 tables in MySQL
The general problem is that I'm merging a new (temporary) database with an old one. I want to keep all the data in the old but copy over any new data from the new. If there is a duplicate, the old should be favored/kept.
My current command is:
INSERT INTO BAT_players
SELECT *
FROM bat2.bat_players
WHERE NOT EXISTS (SELECT 1 FROM BAT_players WHERE BAT_players(UUID) = bat2.bat_players(UUID));
When I run this, I get
Function bat2.bat_players undefined or Function bat.BAT_players undefined
I do not know how to proceed and would appreciate the help.

Columns are accessed using . not parens:
INSERT INTO BAT_players
SELECT *
FROM bat2.bat_players bp2
WHERE NOT EXISTS (SELECT 1
FROM BAT_players bp
WHERE bp.UUID = bp2.UUID
);
Note that the columns have to correspond by position, because you are not explicitly listing them. As a general rule, you want to list all all the columns in an insert:
INSERT INTO BAT_players ( . . . )
SELECT . . .
. . .

I am no familiar with the idea of MySQL,
I worked with SQL Server to be honest but if all the infrastructure are the same and I say IF, then there is a trick to these kinds of transactions between databases and that's simply the phrase dbo.
Like below:
using BAT
Insert into bat_players
SELECT * FROM bat2.dbo.bat_players
and also the rest of your conditions
or
instead of using the phrase using bat you can simply add the dbo to:
Insert into bat.dbo.bat_players
and again the rest of your condition,
just remember to use the dbo before each [table name].
HUGE UPDATE
if you want to access the fields (columns) you have to use . as #Gordon Linoff explained above. For example:
...
Where bat2.dbo.bat_players.UUID = --the condition--

Related

SQL filling a table importing data from another table and math

I am trying to develop software for one of my classes.
It is supposed to create a table contrato where I would fill the info of the clients and how much are they going to pay and how many payments they will make to cancel the contract.
On the other hand I have another table cuotas which should be filled by importing some info from table1 and I'm trying to perform the math and save the payment info directly into the SQL. But it keeps telling me I cant save the SQL because of error #1241
I'm using PHPMyAdmin and Xampp
Here is my SQL code
INSERT INTO `cuotas`(`Ncontrato`, `Vcontrato`, `Ncuotas`) SELECT (`Ncontrato`,`Vcontrato`,`Vcuotas`) FROM contrato;
SELECT `Vcuotaunit` = `Vcontrato`/`Ncuotas`;
SELECT `Vcuotadic`=`Vcuotaunit`*2;
Can you please help me out and fix whatever I'm doing wrong?
Those selects are missing a FROM clause.
So it's unknown from which table or view they have to take the columns.
You could use an UPDATE after that INSERT.
INSERT INTO cuotas (Ncontrato, Vcontrato, Ncuotas)
SELECT Ncontrato, Vcontrato, Vcuotas
FROM contrato;
UPDATE cuotas
SET Vcuotaunit = (Vcontrato/Ncuota),
Vcuotadic = (Vcontrato/Ncuota)*2
WHERE Vcuotaunit IS NULL;
Or use 1 INSERT that also does the calculations.
INSERT INTO cuotas (Ncontrato, Vcontrato, Ncuotas, Vcuotaunit, Vcuotadic)
SELECT Ncontrato, Vcontrato, Vcuotas,
(Vcontrato/Ncuota) as Vcuotaunit,
(Vcontrato/Ncuota)*2 as Vcuotadic
FROM contrato;

Insert into Select command causing exception ParseException line 1:12 missing TABLE at 'table_name' near '<EOF>'

I am 2 days old into hadoop and hive. So, my understanding is very basic. I have a question which might be silly. Question :I have a hive external table ABC and have created a sample test table similar to the table as ABC_TEST. My goal is to Copy certain contents of ABC to ABC_TEST depending on select clause. So I created ABC_TEST using the following command:
CREATE TABLE ABC_TEST LIKE ABC;
Problem with this is:
1) this ABC_TEST is not an external table.
2) using Desc command, the LOCATION content for ABC_TEST was something like
hdfs://somepath/somdbname.db/ABC_TEST
--> On command "hadoop fs -ls hdfs://somepath/somdbname.db/ABC_TEST " I found no files .
--> Whereas, "hadoop fs -ls hdfs://somepath/somdbname.db/ABC" returned me 2 files.
3) When trying to insert values to ABC_TEST from ABC, I have the above exception mentioned in the title. Following is the command I used to insert values to ABC_TEST:
INSERT INTO ABC_TEST select * from ABC where column_name='a_valid_value' limit 5;
Is it wrong to use the insert into select option in Hive? what am I missing? Please help
The correct syntax is "INSERT INTO TABLE [TABLE_NAME]"
INSERT INTO TABLE ABC_TEST select * from ABC where column_name='a_valid_value' limit 5;
I faced exactly the same issue and the reason is the Hive version.
In one of our clusters, we are using hive 0.14 and on a new set up we're using hive-2.3.4.
In hive 0.14 "TABLE" keyword is mandatory to be used in the INSERT command.
However in version hive 2.3.4, this is not mandatory.
So while in hive 2.3.4, the query you've mentioned above in your question will work perfectly fine but in older versions you'll face exception "FAILED: ParseException line 1:12 missing TABLE <>".
Hope this helps.

sql injection operator explanation [duplicate]

Just looking at:
(Source: https://xkcd.com/327/)
What does this SQL do:
Robert'); DROP TABLE STUDENTS; --
I know both ' and -- are for comments, but doesn't the word DROP get commented as well since it is part of the same line?
It drops the students table.
The original code in the school's program probably looks something like
q = "INSERT INTO Students VALUES ('" + FNMName.Text + "', '" + LName.Text + "')";
This is the naive way to add text input into a query, and is very bad, as you will see.
After the values from the first name, middle name textbox FNMName.Text (which is Robert'); DROP TABLE STUDENTS; --) and the last name textbox LName.Text (let's call it Derper) are concatenated with the rest of the query, the result is now actually two queries separated by the statement terminator (semicolon). The second query has been injected into the first. When the code executes this query against the database, it will look like this
INSERT INTO Students VALUES ('Robert'); DROP TABLE Students; --', 'Derper')
which, in plain English, roughly translates to the two queries:
Add a new record to the Students table with a Name value of 'Robert'
and
Delete the Students table
Everything past the second query is marked as a comment: --', 'Derper')
The ' in the student's name is not a comment, it's the closing string delimiter. Since the student's name is a string, it's needed syntactically to complete the hypothetical query. Injection attacks only work when the SQL query they inject results in valid SQL.
Edited again as per dan04's astute comment
Let's say the name was used in a variable, $Name. You then run this query:
INSERT INTO Students VALUES ( '$Name' )
The code is mistakenly placing anything the user supplied as the variable. You wanted the SQL to be:
INSERT INTO Students VALUES ( 'Robert Tables` )
But a clever user can supply whatever they want:
INSERT INTO Students VALUES ( 'Robert'); DROP TABLE Students; --' )
What you get is:
INSERT INTO Students VALUES ( 'Robert' ); DROP TABLE STUDENTS; --' )
The -- only comments the remainder of the line.
As everyone else has pointed out already, the '); closes the original statement and then a second statement follows. Most frameworks, including languages like PHP, have default security settings by now that don't allow multiple statements in one SQL string. In PHP, for example, you can only run multiple statements in one SQL string by using the mysqli_multi_query function.
You can, however, manipulate an existing SQL statement via SQL injection without having to add a second statement. Let's say you have a login system which checks a username and a password with this simple select:
$query="SELECT * FROM users WHERE username='" . $_REQUEST['user'] . "' and (password='".$_REQUEST['pass']."')";
$result=mysql_query($query);
If you provide peter as the username and secret as the password, the resulting SQL string would look like this:
SELECT * FROM users WHERE username='peter' and (password='secret')
Everything's fine. Now imagine you provide this string as the password:
' OR '1'='1
Then the resulting SQL string would be this:
SELECT * FROM users WHERE username='peter' and (password='' OR '1'='1')
That would enable you to log in to any account without knowing the password. So you don't need to be able to use two statements in order to use SQL injection, although you can do more destructive things if you are able to supply multiple statements.
No, ' isn't a comment in SQL, but a delimiter.
Mom supposed the database programmer made a request looking like:
INSERT INTO 'students' ('first_name', 'last_name') VALUES ('$firstName', '$lastName');
(for example) to add the new student, where the $xxx variable contents was taken directly out of an HTML form, without checking format nor escaping special characters.
So if $firstName contains Robert'); DROP TABLE students; -- the database program will execute the following request directly on the DB:
INSERT INTO 'students' ('first_name', 'last_name') VALUES ('Robert'); DROP TABLE students; --', 'XKCD');
ie. it will terminate early the insert statement, execute whatever malicious code the cracker wants, then comment out whatever remainder of code there might be.
Mmm, I am too slow, I see already 8 answers before mine in the orange band... :-) A popular topic, it seems.
TL;DR
-- The application accepts input, in this case 'Nancy', without attempting to
-- sanitize the input, such as by escaping special characters
school=> INSERT INTO students VALUES ('Nancy');
INSERT 0 1
-- SQL injection occurs when input into a database command is manipulated to
-- cause the database server to execute arbitrary SQL
school=> INSERT INTO students VALUES ('Robert'); DROP TABLE students; --');
INSERT 0 1
DROP TABLE
-- The student records are now gone - it could have been even worse!
school=> SELECT * FROM students;
ERROR: relation "students" does not exist
LINE 1: SELECT * FROM students;
^
This drops (deletes) the student table.
(All code examples in this answer were run on a PostgreSQL 9.1.2 database server.)
To make it clear what's happening, let's try this with a simple table containing only the name field and add a single row:
school=> CREATE TABLE students (name TEXT PRIMARY KEY);
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "students_pkey" for table "students"
CREATE TABLE
school=> INSERT INTO students VALUES ('John');
INSERT 0 1
Let's assume the application uses the following SQL to insert data into the table:
INSERT INTO students VALUES ('foobar');
Replace foobar with the actual name of the student. A normal insert operation would look like this:
-- Input: Nancy
school=> INSERT INTO students VALUES ('Nancy');
INSERT 0 1
When we query the table, we get this:
school=> SELECT * FROM students;
name
-------
John
Nancy
(2 rows)
What happens when we insert Little Bobby Tables's name into the table?
-- Input: Robert'); DROP TABLE students; --
school=> INSERT INTO students VALUES ('Robert'); DROP TABLE students; --');
INSERT 0 1
DROP TABLE
The SQL injection here is the result of the name of the student terminating the statement and including a separate DROP TABLE command; the two dashes at the end of the input are intended to comment out any leftover code that would otherwise cause an error. The last line of the output confirms that the database server has dropped the table.
It's important to notice that during the INSERT operation the application isn't checking the input for any special characters, and is therefore allowing arbitrary input to be entered into the SQL command. This means that a malicious user can insert, into a field normally intended for user input, special symbols such as quotes along with arbitrary SQL code to cause the database system to execute it, hence SQL injection.
The result?
school=> SELECT * FROM students;
ERROR: relation "students" does not exist
LINE 1: SELECT * FROM students;
^
SQL injection is the database equivalent of a remote arbitrary code execution vulnerability in an operating system or application. The potential impact of a successful SQL injection attack cannot be underestimated--depending on the database system and application configuration, it can be used by an attacker to cause data loss (as in this case), gain unauthorized access to data, or even execute arbitrary code on the host machine itself.
As noted by the XKCD comic, one way of protecting against SQL injection attacks is to sanitize database inputs, such as by escaping special characters, so that they cannot modify the underlying SQL command and therefore cannot cause execution of arbitrary SQL code. This can be done at the application level, and some implementations of parameterized queries operate by sanitizing input.
However, sanitizing inputs at the application level may not stop more advanced SQL injection techniques. For example, there are ways to circumvent the mysql_real_escape_string PHP function. For added protection, many database systems support prepared statements. If properly implemented in the backend, prepared statements can make SQL injection impossible by treating data inputs as semantically separate from the rest of the command.
Say you naively wrote a student creation method like this:
void createStudent(String name) {
database.execute("INSERT INTO students (name) VALUES ('" + name + "')");
}
And someone enters the name Robert'); DROP TABLE STUDENTS; --
What gets run on the database is this query:
INSERT INTO students (name) VALUES ('Robert'); DROP TABLE STUDENTS --')
The semicolon ends the insert command and starts another; the -- comments out the rest of the line. The DROP TABLE command is executed...
This is why bind parameters are a good thing.
A single quote is the start and end of a string. A semicolon is the end of a statement. So if they were doing a select like this:
Select *
From Students
Where (Name = '<NameGetsInsertedHere>')
The SQL would become:
Select *
From Students
Where (Name = 'Robert'); DROP TABLE STUDENTS; --')
-- ^-------------------------------^
On some systems, the select would get ran first followed by the drop statement! The message is: DONT EMBED VALUES INTO YOUR SQL. Instead use parameters!
The '); ends the query, it doesn't start a comment. Then it drops the students table and comments the rest of the query that was supposed to be executed.
In this case, ' is not a comment character. It's used to delimit string literals. The comic artist is banking on the idea that the school in question has dynamic sql somewhere that looks something like this:
$sql = "INSERT INTO `Students` (FirstName, LastName) VALUES ('" . $fname . "', '" . $lname . "')";
So now the ' character ends the string literal before the programmer was expecting it. Combined with the ; character to end the statement, an attacker can now add (inject) whatever sql they want. The -- comment at the end is to make sure any remaining sql in the original statement does not prevent the query from compiling on the server.
FWIW, I also think the comic in question has an important detail wrong: if you sanitize your database inputs, as the comic suggests, you're still doing it wrong. Instead, you should think in terms of quarantining your database inputs, and the correct way to do this is via parameterized queries/prepared statements.
The writer of the database probably did a
sql = "SELECT * FROM STUDENTS WHERE (STUDENT_NAME = '" + student_name + "') AND other stuff";
execute(sql);
If student_name is the one given, that does the selection with the name "Robert" and then drops the table. The "-- " part changes the rest of the given query into a comment.
The ' character in SQL is used for string constants. In this case it is used for ending the string constant and not for comment.
This is how it works:
Lets suppose the administrator is looking for records of student
Robert'); DROP TABLE STUDENTS; --
Since the admin account has high privileges deleting the table from this account is possible.
The code to retrieve user name from request is
Now the query would be something like this (to search the student table)
String query="Select * from student where username='"+student_name+"'";
statement.executeQuery(query); //Rest of the code follows
The resultant query becomes
Select * from student where username='Robert'); DROP TABLE STUDENTS; --
Since the user input is not sanitized, The above query has is manipulated into 2 parts
Select * from student where username='Robert');
DROP TABLE STUDENTS; --
The double dash (--) will just comment out remaining part of the query.
This is dangerous as it can nullify password authentication, if present
The first one will do the normal search.
The second one will drop the table student if the account has sufficient privileges (Generally the school admin account will run such query and will have the privileges talked about above).
You don't need to input form data to make SQL injection.
No one pointed this out before so through I might alert some of you.
Mostly we will try to patch forms input. But this is not the only place where you can get attacked with SQL injection. You can do very simple attack with URL which send data through GET request;
Consider the fallowing example:
show something
Your url would look
http://yoursite.com/show?id=1
Now someone could try something like this
http://yoursite.com/show?id=1;TRUNCATE table_name
Try to replace table_name with the real table name. If he get your table name right they would empty your table! (It is very easy to brut force this URL with simple script)
Your query would look something like this...
"SELECT * FROM page WHERE id = 4;TRUNCATE page"
Example of PHP vulnerable code using PDO:
<?php
...
$id = $_GET['id'];
$pdo = new PDO($database_dsn, $database_user, $database_pass);
$query = "SELECT * FROM page WHERE id = {$id}";
$stmt = $pdo->query($query);
$data = $stmt->fetch();
/************* You have lost your data!!! :( *************/
...
Solution - use PDO prepare() & bindParam() methods:
<?php
...
$id = $_GET['id'];
$query = 'SELECT * FROM page WHERE id = :idVal';
$stmt = $pdo->prepare($query);
$stmt->bindParam('idVal', $id, PDO::PARAM_INT);
$stmt->execute();
$data = $stmt->fetch();
/************* Your data is safe! :) *************/
...

Getting data from simple SELECT using twisted.enterprise.adbapi

I am able to do mySQL data insert using following,
from twisted.enterprise.adbapi import ConnectionPool
.
.
self.factory.pool.runOperation ('insert into table ....')
But, somehow unable to figure out how to do a simple select from an adbapi call to mySQL like following,
self.factory.pool.runOperation('SELECT id FROM table WHERE name = (%s)',customer)
How do I retrieve the id value from this partilcar call? I was working OK with plain python but somehow really fuzzed up with the twisted framework.
Thanks.
runOperation isn't for SELECT statements. It is for statements that do not produce rows, eg INSERT and DELETE.
Statements that produce rows are supported by runQuery. For example:
pool = ...
d = pool.runQuery("SELECT id FROM table WHERE name = (%s)", (customer,))
def gotRows(rows):
print 'The user id is', rows
def queryError(reason):
print 'Problem with the query:', reason
d.addCallbacks(gotRows, queryError)
In this example, d is an instance of Deferred. If you haven't encountered Deferreds before, you definitely want to read up about them: http://twistedmatrix.com/documents/current/core/howto/defer.html

Select from all tables

I have a lot of tables in my data base all with same structure. I want to select from all tables without having to list them all like so:
SELECT name FROM table1,table2,table3,table4
And I tried but this doesn't work:
SELECT name FROM *
Is there a way to select all tables in a database without listing each table in the query?
i am working on a online file browser, each directory has its own table
It is very unuseful due to one reason: when you have about 200 files (this situation is real, yeah?) you have about 200 tables. And if there are about thousand files in each directory.. etc. In some time you will either have slow processing while selecting from your database either have to buy more server resources.
I think you should change your database structure: just begin from adding parent_folder_id column to your table, after this you can put all your rows (files and directories -- because directory is a file too -- here you can add type column to determine this) into the one table.
As far as I know there are no such wildcards to select from *all tables. I would recommend writing a view and then call that view instead (it will save you writing out the names every time) – VoodooChild
That means you should not have a lot of tables with same structure at all.
But just one table with a field to distinguish different kinds of data, whatever it is.
Then select all would be no problem.
I found a solution, but I would still like to know if there is a simpler way or a better solution.
But here's what I came up with:
$tables = mysql_query("show tables");
$string = '';
while ($table_data = mysql_fetch_row($tables)){
$string.=$table_data[0].',';
}
$ALL_TABLES = substr($string,0,strlen($string)-1);
$sql="SELECT name FROM $ALL_TABLES ";
Sounds like you want to UNION together each table, so you get the results as if they were one big table. You'll need to write out the query in full like
SELECT * FROM table1 UNION SELECT * FROM table2 UNION ... SELECT * FROM tableN
Copy & paste may be your friend here.
I'm curious as to why you have lots of different tables with the same structure?
You can generate SELECT by cursor like this code
and find all result step by step in sql server:
--Author: Ah.Ghasemi
Declare #Select sysname;
DECLARE A CURSOR
FOR Select 'select ' + '*' + ' from ' + name
from sys.tables
--Where name like 'tbl%'
Order by name
OPEN A
FETCH NEXT FROM A INTO #Select
While (##FETCH_STATUS <>-1)
Begin
exec sp_executesql #Select
FETCH NEXT FROM A INTO #Select;
End
close A
Deallocate A
Please let us know if the problem is not resolved.
I hope you for the best