I have postponed writing SQL code for university, and now that I want to start learning it, I have no idea how to.
In C I'd define headers and begin with coding main, but in SQL classes all I have is a plain example
CREATE TABLE Sailors(
sid INTEGER,
sname VARCHAR(30) NOT NULL,
rating INTEGER DEFAULT 0,
age REAL DEFAULT 18
)
and some commands for using the table I created.
My questions are: How is a correct script supposed to look? How do I run it to create a database? (MySQL) How do I use MySQL to run scripts and where do I type commands in real time to do stuff I haven't scripted?
I just can't wrap my head around it. All tutorials I've seen use a terminal I can't find, or another I did find and I can't use because I get errors using any command (can't create file in some directory and some modules report errors so it shuts down)
The following is a very vague description of Mysql to get an idea of it:
Mysql (or SQL) is separated in 3 types of language:
DML : Data Manipulation Language
DDL : Data Definition Language
DCL : Data Control Language
Read about them to find out which kind of command belongs where.
You will find out that you almost exclusively need DML and DDL to work with Data in MySQL. While DCL is mostly used to keep the database running, control user privileges , etc.
Also when running code there will be only one command of your script executed at a time without a possibility to point somewhere else in your script.
Loops and Cursors can be used , but have to be stored in a special form of script called stored procedure. Usually you execute your code in a sequence without a code based relation between the different commands (the relation comes from the context of the commands).
Get Data into your Database:
(Consider installing the community edition for MySQL if you have problems running MySQL correctly)
To get Data into your Database , you should import data from files into your database. The MySQL-GUIs available (Workbench, Toad, Navicat, HeidSQL...) usually provide an Import Wizard that makes it easy to import Data from all kind of Files (txt, Excel, Database Files ..).
You can create an excel spreadsheet and import it into your database for example.
here is a picture of the Workbench SQL Editor:
https://dev.mysql.com/doc/workbench/en/images/wb-getting-started-tutorial-adding-data-movies.png
Workbench (or any other GUI) will be your IDE. Getting into it will answer many of your questions.
Regarding the correct script:
A complete MySQL command is called a query.
A Query is defined by a ; at the end (default) .
A chain of MySQL commands is called a script.
Therefore, a correct script consists of correct querys.
To solve more complex problems, use stored procedures in MySQL (this should come close to your usage of the word script).
some MYSQL commands you will have to be familiar with:
select
update
insert into
delete
create table
drop table
alter table
You have a lot to read. But make sure that your Database is running and you have some data in it to test code. As you already have programming experience, you should understand this really fast with the right setup.
Related
I have a .sql file from Oracle which contains create table/index statements and a lot of insert statements(around 1M insert).
I can manually modify the create table/index part(not too much), but for the insert part there are some Oracle functions like to_date.
I know MySql has a similar function STR_TO_DATE but the usage of the parameter is different.
I can connect to MySQL, but the .sql file is the only thing I got from Oracle.
Is there any way I can import this Oracle .sql file into MySQL?
Thanks.
Although the above job can be done by manually editing the script appropriately however there are products available which can be of use. Refer to the link for more information on one such product.
P.S. I am not affiliated in any way to the product
Since you mention about insert script basically i think you will be inserting data for this you can use any ETL tool, like open source tool like Pentaho data integrator, pretty simple to do, just search table to table transformation from different database connection on youtube to learn you should be able to connect to both mysql and oracle database else this wont help, but all the table structures you should create manually in the source database for data - you can just load it using ETL, no need to edit for every single line of insert if its more than 100 may be its very painful thing to do.
I am very new to mysql and I have I a situation where I need to update all my stored procedure in all my database. for example I have 10 database just say:
client_1,client_2,client_3,.....client_10.
Every database have same stored procedure just say:
proc_1,proc_2,proc_3,proc_4.
So if I made any changes to one of my stored procedure then it should get updated in all other database So that I don't have to do it manually.
I know the similar question have been asked but I am looking for some different approach. So what I want is some kind of mysql query or something like that in which we will pass the name of the database like:
client_1, client_3, client_8
and changes will only made to this databases.
I am using heidiSql- 10.2 with MySQL 5.6.
Thanks.
I am not entirely sure what you are wanting to do but I think you want something like this. First save the definition of your stored procedure to a file. Make sure it doesn't contain an schema references like client1.tableA. You want it to be able to run in any copy of your schema correctly. Be sure to follow the syntax rules defined by MySQL
Defining Stored Programs
Then once the stored procedure is saved you can use the mysql command line to run it for each client you want to update.
You would first connect to the database server using the mysql command line. Then issue a USE command to activate the first client database. Then run the script using the SOURCE command. See MySQL Batch Commands Then repeat for each client.
USE client1;
source c:\temp\storedProcedure.sql
USE client2;
source c:\temp\storedProcedure.sql
If this is not exactly what you needed hopefully it gives you some ideas to get you what you need.
Note that you could do the connection to the database and execute these commands via batch file instead of manually if you wanted to.
There are no statements in MySQL that create/drop/alter multiple procedures at once. You can only change one procedure at a time.
You can write an SQL script that includes a series of statements. But it's up to you to write that script.
You may write some script in Python (or other favorite language) that outputs the SQL script.
I don't know HeidiSQL, but I doubt it has any facility to apply the same change to many procedures. Nor does any other MySQL client that I'm aware of.
I have tried to find an answer for this elsewhere but cannot, I hope someone can help me.
I am trying to import the MySQL sample database into Oracle SQL Developer.
I have an existing database/connection I want to dump it into. I have created an empty table named classicmodels in my existing connection. Yes that name is only 1 table within the sample db, correct. Ignore the error in naming convention.
When I R-click on it and try 'import data' I cannot import a .sql file, I can only do it with XL, CSV, etc.
When I try and run a script it found on dba.stackexchange
#\path\mysqlsampledatabase.sql , I get a series of 'please provide substitution value' messages, which does not make sense to me given that I am importing a database which is built for SQL (ie what reason is there to substitute).
Pictures below:
The 'UnseenCollection' is a single table I imported as a csv file. I need to import the mysqlsampledatabase file such that it shows up the same way, I can access all tables within the sample db.
Anyone can help I would appreciate it. I need the end result to be the entire mysqlsampledatabase to populate within the 'classicmodels' node.
Thank you.
connect to MySQL
connect to Oracle
for a single MySQL table, right-click, 'Copy to Oracle'
for a few tables, select, drag and drop onto Oracle connection (requires newer version of SQL Developer)
for an entire MySQL database, use the migration project feature
What's the best way to save my MySQL data model and automatically apply changes to my development database server as they are made (or at least nightly)?
For example, today I'm working on my project and create this table in my database, and save the statement to SQL file to deploy to production later:
create table dog (
uid int,
name varchar(50)
);
And tomorrow, I decide I want to record the breed of each dog too. So I change the SQL file to read:
create table dog (
uid int,
name varchar(50),
breed varchar(30)
);
That script will work in production for the first release, but it won't help me update my development database because ERROR 1050 (42S01): Table 'dog' already exists. Furthermore, it won't work in production if this change was made after the first release. So I really need to ALTER the table now.
So now I have two concerns:
Is this how I should be saving my
data model (a bunch of create
statements in a SQL file), and
How
should I be applying changes like
this to my database?
My goal is to release changes accurately and enable continuous integration. I use a tool called DDLSYNC do find and apply difference in an Oracle database, but I'm not sure what similar tools exist for MySQL.
At work, we developed a small script to manage our database versioning. Every change to any table or set of data gets it's own SQL file.
The files are numbered sequentially. We keep track of which update files have been run by storing that information in the database. The script inserts a row with the filename when the file is about to be executed, and updates the row with a completion timestamp when the execution finishes. This is wrapped inside a transaction. (It's worth remembering that DDL commands in MySQL can not occur within a transaction. Any attempt to perform DDL in a transaction causes an implicit commit.)
Because the SQL files are part of our source code repository, we can make running the update script part of the normal rollout process. This makes keeping the database and the code in sync easy as pie. Honestly, the hardest part is making sure another dev hasn't grabbed the next number in a pending commit.
We combine this update system with an (optional) nightly wipe of our dev database, replacing the contents with last night's live system backup. After the backup is restored, the update gets run, with any pending update files getting run in the process.
The restoration occurs in such a way that only tables that were in the live database get overwritten. Any update that adds a table therefore also has to be responsible for only adding it if it doesn't exist. DROP TABLE IF EXISTS is handy. Unfortunately not all databases support that, so the update system also allows for execution of scripts written in our language of choice, not just SQL.
All of this in about 150 lines of code. It's as easy as reading a directory, comparing the contents to a table, and executing anything that hasn't already been executed, in a determined order.
There are standard tools for this in many frameworks: Rails has something called Migrations, something that's easily replicated in PHP or any similar language.
I have reached the limit of RAM in analyzing large datasets in R. I think my next step is to import these data into a MySQL database and use the RMySQL package. Largely because I don't know database lingo, I haven't been able to figure out how to get beyond installing MySQL with hours of Googling and RSeeking (I am running MySQL and MySQL Workbench on Mac OSX 10.6, but can also run Ubuntu 10.04).
Is there a good reference on how to get started with this usage? At this point I don't want to do any sort of relational databasing. I just want to import .csv files into a local MySQL database and do the subsetting in with RMySQL.
I appreciate any pointers (including "You're way off base!" as I'm new to R and newer to large datasets... this one's around 80 mb)
The documentation for RMySQL is pretty good - but it does assume that you know the basics of SQL. These are:
creating a database
creating a table
getting data into the table
getting data out of the table
Step 1 is easy: in the MySQL console, simply "create database DBNAME". Or from the command line, use mysqladmin, or there are often MySQL admin GUIs.
Step 2 is a little more difficult, since you have to specify the table fields and their type. This will depend on the contents of your CSV (or other delimited) file. A simple example would look something like:
use DBNAME;
create table mydata(
id INT(11) NOT NULL AUTO_INCREMENT PRIMARY KEY,
height FLOAT(3,2)
);
Which says create a table with 2 fields: id, which will be the primary key (so has to be unique) and will autoincrement as new records are added; and height, which here is specified as a float (a numeric type), with 3 digits total and 2 after the decimal point (e.g. 100.27). It's important that you understand data types.
Step 3 - there are various ways to import data to a table. One of the easiest is to use the mysqlimport utility. In the example above, assuming that your data are in a file with the same name as the table (mydata), the first column a tab character and the second the height variable (with no header row), this would work:
mysqlimport -u DBUSERNAME -pDBPASSWORD DBNAME mydata
Step 4 - requires that you know how to run MySQL queries. Again, a simple example:
select * from mydata where height > 50;
Means "fetch all rows (id + height) from the table mydata where height is more than 50".
Once you have mastered those basics, you can move to more complex examples such as creating 2 or more tables and running queries that join data from each.
Then - you can turn to the RMySQL manual. In RMySQL, you set up the database connection, then use SQL query syntax to return rows from the table as a data frame. So it really is important that you get the SQL part - the RMySQL part is easy.
There are heaps of MySQL and SQL tutorials on the web, including the "official" tutorial at the MySQL website. Just Google search "mysql tutorial".
Personally, I don't consider 80 Mb to be a large dataset at all; I'm surprised that this is causing a RAM issue and I'm sure that native R functions can handle it quite easily. But it's good to learn new skill such as SQL, even if you don't need them for this problem.
I have a pretty good suggestion. For 80MB use SQLite. SQLite is a super public domain, lightweight, super fast file-based database that works (almost) just like a SQL database.
http://www.sqlite.org/index.html
You don't have to worry about running any kind of server or permissions, your database handle is just a file.
Also, it stores all data as a string, so you don't even have to worry about storing the data as types (since all you need to do is emulate a single text table anyway).
Someone else mentioned sqldf:
http://code.google.com/p/sqldf/
which does interact with SQLite:
http://code.google.com/p/sqldf/#9._How_do_I_examine_the_layout_that_SQLite_uses_for_a_table?_whi
So your SQL create statement would be like this
create table tablename (
id INT(11) INTEGER PRIMARY KEY,
first_column_name TEXT,
second_column_name TEXT,
third_column_name TEXT
);
Otherwise, neilfws' explanation is a pretty good one.
P.S. I'm also a little surprised that your script is choking on 80mb. It's not possible in R to just seek through the file in chunks without opening it all up in memory?
The sqldf package might give you an easier way to do what you need: http://code.google.com/p/sqldf/. Especially if you are the only person using the database.
Edit: Here is why I think it would be useful in this case (from the website):
With sqldf the user is freed from having to do the following, all of which are automatically done:
database setup
writing the create table statement which defines each table
importing and exporting to and from the database
coercing of the returned columns to the appropriate class in common cases
See also here: Quickly reading very large tables as dataframes in R
I agree with what's been said so far. Though I guess getting started with MySQL (databases) in general is not a bad idea for the long if you are going to deal with data. I mean I checked your profile which says finance PhD student. I don't know if that means quant. finance, but it is likely that you will come across really large datasets in your career. I you can afford some time, I would recommend to learn something about databases. It just helps.
The documentation of MySQL itself is pretty solid and you can a lot of additional (specific) help here at SO.
I run MySQL with MySQL workbench on Mac OS X Snow Leopard too. So here´s what helped me to get it done comparatively easy.
I installed MAMP , which gives my an local Apache webserver with PHP, MySQL and the MySQL tool PHPmyadmin, which can be used as a nice webbased alternative for MySQL workbench (which is not always super stable on a Mac :) . You will have a little widget to start and stop servers and can access some basic configuration settings (such as ports through your browser) . It´s really one-click install here.
Install the Rpackage RMySQL . I will put my connection string here, maybe that helps:
Create your databases with MySQL workbench. INT and VARCHAR (for categorical variables that contain characters) should be the field types you basically need at the beginning.
Try to find the import routine that works best for you. I don't know if you are a shell / terminal guy – if so you'll like what was suggested by neilfws. You could also use LOAD DATA INFILE which is I prefer since it's only one query as opposed to INSERT INTO (line by line)
If you specify the problems that you have more accurately, you'll get some more specific help – so feel free to ask ;)
I assume you have to work a lot with time series data – there is a project (TSMySQL) around that use R and relational databases (such as MySQL, but also available for other DBMS) to store time series data. Besides you can even connect R to FAME (which is popular among financers, but expensive). The last paragraph is certainly nothing basic, but I thought it might help you to consider if it´s worth the hustle to dive into it a little deeper.
Practical Computing for Biologists as a nice (though subject-specific) introduction to SQLite
Chapter 15. Data Organization and Databases