First up, this might be the wrong place to ask this question.. So, sincere apologies for that!
I have two MySQL Tables as follows:
First Table: employee_data
id name address phone_no
1 Mark Some Street 647-981-1512
2 Adam Some Street 647-981-1214
3 John Some Street 647-981-1452
Second Table: employee_wages
id employee_id wages start_date
1 3 $15 12 March 2007
2 1 $20 10 Oct 2008
3 2 $18 2 June 2006
I know, both these tables can be combined into one and there is no need to split this data into two tables. But, what i'm working on requires this data to be separate and in two different tables.
Now, previously my company used to handle all this data in Excel sheets and they followed the conventional method of having these two tables combined into one sheet as follows:
Excel Sheets
id name wages start_date
1 Mark $20 10 Oct 2008
2 Adam $18 2 June 2006
3 John $15 12 March 2007
Now, the objective is to Export the data from Excel sheets into MySQL Tables.
As you can notice employee_data.id is linked to employee_wages.employee_id
How can i replace the values in the Excel Sheet 'name' column so that they represent the actual unique ID they're given in the employee_data.id column..
May be i can do it with PHP/MySQL or i can get this done in VB Script.. BUt, I'm not an expert in VB Script..
Any help will be much appreciated..
Thanks!
Save your Excel data as a csv file (In Excel 2007 using Save As)
Check the saved file using a text editor such as Notepad to see what it actually looks like, i.e. what delimiter was used etc.
Start the MySQL Command Prompt (I’m lazy so I usually do this from the MySQL Query Browser – Tools – MySQL Command Line Client to avoid having to enter username and password etc.)
Enter this command:
LOAD DATA LOCAL INFILE ‘C:\\temp\\yourfile.csv’ INTO TABLE database.table FIELDS TERMINATED BY ‘;’ ENCLOSED BY ‘”‘ LINES TERMINATED BY ‘\r\n’ (field1, field2);
[Edit: Make sure to check your single quotes (') and double quotes (") if you copy and paste this code - it seems WordPress is changing them into some similar but different characters]
Done!
use the vlookup function in excel.
import the data into excel
use vlookup to combine what you need
You can query MySQL from Excel, this example uses INSERT: Excel VBA: writing to mysql database
A recordset can be written to Excel with CopyRecordset : http://support.microsoft.com/kb/246335
Related
I am looking for solution to taking a template excel document and embedding it onto my website. After this users will use the template filling out certain cells. Is it possible for me to scrape a embedded excel sheet and use POST methods to send the data to my SQL Server database.
I want to embed the template because I need to change it from time to time, which means I love to have it easily changed within my code using the embed and iframe.
I have looked online and haven't found any solid documentation that this is possible.
I suggest that you start by thinking about the differences between data in an Excel spreadsheet and data in a normalised SQL Server database.
For example, suppose you have a spreadsheet that you use to record downloads of "digital items" such as pdf files, word documents, .zip files, and so on.
Each column represents a date and each row represents a digital item.
So lest say Row 4 is for "The Cat Story.pdf" and Column 3 is for "15 February 2019. and Column 4 is for 16 February 2019 and so on. The value in Cell Row 4-Column 3 is "206", and the value in Cell Row 4-Column 4 is "327".
These two cells represent the following facts:
Fact 1: The pdf "Cat Story" was downloaded 206 times on 15 Feb 2019.
Fact 2: The pdf "Cat Story" was downloaded 327 times on 16 Feb 2019.
In an SQL Server database you could store such facts in a simple table called "Downloads" with Column 1=Item name, Column 2=Date and Column 3 = Number of downloads.
My longwinded explanation is to make it clear that in your case you are using an Excel spreadsheet as a tool to display data whereas the purpose of a well-designed SQL Server database is to store facts without any duplications.
After you have decided on a normalised design for your SQL Server database, you can link your Excel spreadsheet directly to the data in your database.
I am trying to create a simple chart on Data Studio that pulls values from a MySQL table. The values are:
John 3
Peter 2
Mary 1
John 2
However on Data Studio, the data comes out Aggregated as:
John 5
Peter 2
Mary 1
I need to see individual lin items, but Data Studio seems to insist on aggregating them. Selecting NONE for aggregation type causes Google to declare the field as invalid.
How can I directly display line by line data from MySQL table without Data Studio trying to aggregate?
I realize this question is rather old and Data Studio has changed a lot since then.
You can also set the aggregation type to "None" in the Data Source for the second column or when the column is added to the table, edit it by clicking on the pencil icon to the left and set the aggregation there.
Adding a unique column (last name) to the data prevented aggregation from Data Studio
I have an issue importing some CSV/TXT files.
Here at the company we receive files from other sources (companies). Some of these files sometimes come partially broken.
For example, a file containing 6 columns (id, name, city, state, zipCode, phone) and 2 million lines. The first 10.000 lines of that file are OK. But in the middle of the file instead of 6 columns, it has 5 or even 7 columns.
It seems like somebody "merged" several files into this one and did not pay attention to the number of columns. So when I import it to my MySql database table, the data comes very messy due to the columns being broken. The zipCode records show up on the field state and so on.
I was wondering how to scan such file before importing it to my DB, something like counting the ";" delimiters of each line. Would it be done using Regex or what would be the best option for that?
My program is written in Lazarus/Pascal.
I would read the file line by line and check the columns.
If a line respects the expected columns (count, , copy it in another file (input_OK.csv).
If it doesn't dump it in a broken lines file (input_KO.csv).
Study input_KO.csv errors, correct them then import the corrected file into the database.
IMO, a regex will take long here.
I am coding in VBA to write a macro that essentially does 2 things:
Connects to a MySQL database MYDATABASE, which has one table
Loop and compare all of the values in a column of an Excel Spreadsheet and compare to a column MyColumn of a MySQL table MYTable
To be specific, my table has three values. An ID, a name, and a number. In my excel sheet, I would have much more information, including ID, address, city, and number. However, there would be no name.
I want to loop through the values of the number column in my spreadsheet, and compare the numbers to numbers in the MYTable table. Then, it would pull all associated names to those numbers and place those numbers in an adjacent column in my excel sheet. For the purpose of this macro... the column I want to loop through is column C.
I found this question: How can VBA connect to MySQL database in Excel?
which explains how to connect to VBA.
So how would I go about this?
I would load the excel data into a second MySQL table and then write a simple program to do the comparison between the 2 MySQL tables.
I have a number of CSV files with hundreds of columns and about 50,000 rows (when opened in Excel). The column headers are almost identical however some column headers may vary from one CSV file to the next, as an example below:-
CSV1
Name Surname DOB
John Smith 31/01/1989
CSV2
Name Age Surname Address DOB
Paul 29 Jones 123 Smith St 30/12/1981
CSV3
Name Surname Address Telephone
Mick Jones 123 Paul St 0123456
Is there any way I can merge all of these into one big CSV file, appending the headers so that in the one main CSV, I would have the headers "Name, Surname, DOB, Age, Address, Telephone" for example and then the respective entries from each CSV falling within their respective column heading. The reason I want to do this is to then populate the information into a big MySql / Sql Server DB table and so it appears easier to do it all initially as one big CSV before importing.
Any suggestions?
Import them into three temporary tables and then merge them into one table using joins on name surname and DOB. Otherwise the data will get all mixed up.
Manual method (bear with me, just giving an idea of the algorithm):
Generate a final list of columns that includes all possible headers in all CSV's.
Open each spreadsheet, one at a time. For each spreadsheet:
Click and drag the headers and insert missing columns so they all match your list from #1
Save the file, and repeat back to #2
Combine all the spreadsheets into a single spreadsheet.
Import.
If you are going to automate this, you will take roughly the same steps. You need a way to determine what all columns are possible, then put the CSV's in the right format and combine them, either in spreadsheet/CSV format, or import them as a bunch of temp tables, and INSERT...SELECT to re-arrange the columns where they belong.
What languages/technologies do you have available to you for the automation? .NET? Java? PHP? How often will this process occur, and how automated does it have to be? Is it a daily process, or weekly, or only going to happen once? How many spreadsheets roughly?