Update and add data in sql from multiple excel files - mysql

Let's say I have the following sheet on an excel file:
KEY | ID | Account | Value
01002 | 01 | 002 | 10
As you can see, "KEY" is result of combining "ID" and "Account"
Now, I know how to import a table into an SQL form. I also know how to replace it. But I'm having trouble to add and update SQL table from an excel file.
For example,
If after adding this to an SQL table:
KEY | ID | Account | Value
01003 | 01 | 003 | 10
02008 | 02 | 008 | 11
I then update this :
KEY | ID | Account | Value
01003 | 01 | 003 | 14
02009 | 02 | 009 | 10
key 01003 will be updated with the new value, while key 02009 will be added as a new entry. key 02008 will stay as it is.
Is there any way to do this?
If so, is there any way to run this on multiple workbooks (let's say, multiple workbooks in a folder)
I'm fine with VBA (excel add-in)
One way I could think about the latter one is using the same macro that was used to merge several workbooks into one, but replacing the query to update the excel file with whatever query that can do the above thing (update or add)

I would recommend checking out Sequel pro. It has a great import function that walks you through everything. As well as set the delimiters nice and easy!
I believe it will solve your key issue.
If you need help further comment and ill help you as much as I can
http://www.sequelpro.com/

Related

update target table based on data source table if empty

Possibly a duplicate been searching for the specific answer i need but couldnt find it. I have two simple tables
Source :
| Id | companyName | adress |
|----|-------------|---------|
| 1 | aquatics | street1 |
| 2 | rivers | street2 |
target :
| Id | nameCompany | companyAdress |
|----|-------------|----------------|
| 1 | aquatics | street1 |
| 2 | rivers | |
I simplified the matter, I have two sets of data the source table is extern data en i as a dev want to update my table with the extern data.
So we see that in the source everything is filled in.
I miss some info. In this case i miss the adress.
How can i run a query that checks: your row is incomplete. Lemme update this target row with the source data.
Only problem is. Extern data uses a different name voor the same columns as i have.
Been a couple of days with mysql. So pls try to explain noob friendly tried some thing but i couldnt figure it out
Below query updates companyAdress on target table if adress column on Source table is not equal to companyAdress on target table.
If you only need to update the empty values for target.companyAdress you should change the condition where s.adress <> t.companyAdress to where t.companyAdress =''
update target t
inner join `Source` s
on t.nameCompany=s.companyName
set t.companyAdress=s.adress
where s.adress <> t.companyAdress ;
Demo: https://www.db-fiddle.com/f/pB6b5xrgPKCivFWcpQHsyE/0

Avoid Duplicate Records with BeforeChange Table Event

I have a situation in MS Access database that I must prevent duplicate records based on combination of three attributes:
StudentNumber
ColleagueID
TypeOfAttending
So, for one combination (StudentNumber & ColleagueID) I have three types of attending: A, B and C.
Here is an example:
+---------------+-------------+---------------+
| StudentNumber | ColleagueID | AttendingType |
+---------------+-------------+---------------+
| 100 | 10 | A |
| 100 | 10 | B |
| 100 | 10 | C |
| 100 | 11 | A |
| 100 | 11 | B |
| 100 | 11 | C |
| 100 | 11 | C |
+---------------+-------------+---------------+
So last row would not be acceptable.
Does anyone have any idea?
As noted, you could choose all 3 as a PK. Or you can even create a unique index on all 3 columns. These two ideas are thus code free.
Last but least, you could use a Before change macro,and do a search (lookup) in the table to check if the existing record exists. So far, given your information, likely a unique index is the least effort, and does not require you to change the PK to all 3 columns (which as noted is a another solution).
So, you could consider a before change macro. And use this:
Lookup a Record in MyTable
Where Condition = [z].[Field1]=[MyTable].[Field1] And
[z].[Field2]=[MyTable].[Field2] And
[z].[ID]<>[MyTable].[ID]
Alias Z
RaiseError -123
Error Description: There are other rows with this data
So, you can use a data macro, use the before change table macro. Make sure you have the raise error code indented "inside" of the look up code. And note how we use a alias for the look up, since the table name (MyTable) is already in context, and is already the current row of data, so we lookup using "z" as a alias to distinguish between the current row, and that of lookup record.
So, from a learning point of view, the above table macro can be used, but it likely less work and effort to simply setup a uniquie index on all 3 columns.

MySQL - Multiple Rows or JSON

I'm building an app in Laravel and have a design question regarding my MySQL db.
Currently I have a table which defines the skills for all the default characters in my game. Because the traits are pulled from a pool of skills, and have a variable number, one of my tables looks something like this:
+----+--------+---------+-----------+
| ID | CharID | SkillID | SkillScore|
+----+--------+---------+-----------+
| 1 | 1 | 15 | 200 |
| 2 | 1 | 16 | 205 |
| 3 | 1 | 12 | 193 |
| 4 | 2 | 15 | 180 |
+----+--------+---------+-----------+
Note the variable number of rows for any given CharID. With my Base Characters entered, I'm at just over 300 rows.
My issue is storing User's copies of their (customized)characters. I don't think storing 300+ rows per user makes sense. Should I store this data in a JSON Blob in another table? Should I be looking at a NoSQL solution like Mongo? Appreciate the guidance.
NB: The entire app centers around using the character's different skills. Mostly reporting from them, but users will also be able to update their SkillScore (likely a few times a week).
ps. Should I consider breaking each character out into their own table and tracking user's characters that way? Users won't be able to add/remove the skills from characters, only update them.
TIA.
Your pivot table looks good to me.
I'd consider dropping the ID column (unless you need it), and using a composite primary key:
PRIMARY_KEY(CharID, SkillID)
Primary keys are indexed so you will get efficient lookups.
As for your other suggestions, if you store this in a JSON column, you'll lose the ability to perform joins, and will therefore end up executing more queries.

Access 2007: crosstab view to insert values into a database

I am stuck and need help/advise. I am pretty sure that I am not the first one to run into this problem, but I can't seem to find the answer on the web.
We are collecting all kinds of data from many factories. This is mainly forecasted values of yearly peak production, etc. This data collection is repeated every year.
We currently keep track of this data in Excel, which has the following structure:
Factory | 2010 | 2011 | 2012 | ..
----------------------------------
A | 20 | 30 | 28 | ..
B | | 39 | 55 | ..
In this example factory B just starts production in the year 2011. If we collect data for an additional year, we simply add a column. If forecasted data changes, we simply enter the new values and lose the old ones. You can imagine that this way of working has its limitations: the table becomes rather sparse for missing data. Old values cannot be traced back. There is no reference to the source of the values.
To satisfy our need for a better system, I put my antique knowledge of databases to work. In Access 2007 I created the following structure:
Table: Factories
FacID | FactoryName
---------------------
1 | A
2 | B
Table: Sources
SouID | Source | SourceDate
---------------------------------
1 | DocumentX | Sep. 2009
2 | DocumentY | Jan. 2010
Table: Parameters
ParID | FacID | SouID | ParamType | Year | Value
------------------------------------------------------
1 | 1 | 1 | PeakProduction | 2010 | 20
2 | 1 | 1 | PeakProduction | 2011 | 30
3 | 1 | 1 | PeakProduction | 2012 | 28
4 | 2 | 1 | PeakProduction | 2011 | 39
5 | 2 | 2 | PeakProduction | 2012 | 55
For each new data collection we just add a new source document and append the Parameters table. In this way, we can always revert back to old data. Furthermore, if additional years are collected, there is no need to add additional columns to any table.
Although the actual setup is more complex, above examples is sufficient to illustrate the problem that I am running into: To enter data into the database I would like to have a single form which resembles the original Excel sheet lay-out, i.e.:
Factory | 2010 | 2011 | 2012 |
------------------------------
A | | | |
B | | | |
Of course, this form will have some drop-down menu to select the source document and parameter type ("PeakProduction" in the example).
My question: With crosstab queries it is easy to create such a view based of existing data in the database, however, entering new values is not allowed. What can I do to make this work, and how?
Should I reconsider the design of my database? Should I work with VBA? Link the Access database with Excel sheets?
Thanks!
Where you are dealing with 2 dimensional data that is normalised into a table it is problematic for the user to maintain using Access. My approach to this has been to use the appropriate tool for the job, which looks like Excel in this case. I have the an excel spreadsheet template for data entry. The user enters the data into that. Then in VBA I open the spreadsheet as an embedded object, retrieve the cell contents and insert it into the table. Something like below.
dim myRec as recordset
dim xlApp as Excel.application
dim xlWrksht as Excel.worksheet
set myRec=currentdb.openrecordset("NameOfTable")
set xlApp=createobject("Excel.Application")
set xlWrksht=xlApp.Open("PathOfWorksheet").Worksheets( "WorksheetNumber")
myrec.addnew
myrec.fields("NameOfFields")=xlWrkSht.cells(1,"A")
......
myRec.update

Mysql - backup partial data

Is there an easy way to backup and restore partial data from a mysql database while maintaining the FK constraints?
Say if I have 2 tables
| CustomerId | CustomerName |
-----------------------------
| 12 | Bon Jovi |
| 13 | Seal |
and
| AddressId| CustomerId | City |
---------------------------------------
| 1 | 12 | London |
| 2 | 13 | Paris |
The backup would only take customer 12 and address 1.
My goal is to take a large database from a production server and replicate it locally, but with partial data.
Due to fairly complicated schema, a custom query is not an option. Also I can't rely on the existence of a main table from which one would get the related rows.
Thanks
You could replicate specific customers manually and by adding an FK constraint on the address table replication will fail to insert/update these records.
For replicating specified tables in the db http://dev.mysql.com/doc/refman/5.1/en/replication-options-slave.html#option_mysqld_replicate-do-table .
Use this parameter to silently skip errors on replication http://dev.mysql.com/doc/refman/5.1/en/replication-options-slave.html#sysvar_slave_skip_errors .