I have this assignment that I think someone should be able to help me. I have 5 ACCESS databases wvrapnaoh.accdb, wvrappaul.accdb, ....etc. These databases have about 45 tables each and 15 forms. The good part is the structure, the name and the fields of each table in all the databases are all the same except the data or the records are different. For example I have a stress table in wvrapnoah as well as wvrappaul with the same fields in both tables but different data or records.
So, I need to merge all these five into a new Access database that will have the same structure as the 5 databases but will include the complete data that is all the records from the 5 databases merged into this new database.The same applies to even the 15 forms. It does not seem to be having a primary key I guess. I was planning to add a field for each table that would give me the name of the database as well from which it was merged. Example I will add a DBName field in Wvrapnoah in all the tables and add the name Noah in that field for all the records in each table. I basically need to automate this code.
I need a script (VBA or anything) so that the guys creating these databases can just run this script the next time and merge the databases.
Talking about the 'table' part of the problem:
Questions
Are the databases / table names defined or you don't know them?
Are you able to use linked tables?
I believe the straightforward way to merge all of them is to link all tables into a single access DB and then run a UNION ALL query. It would be something like this:
SELECT "HANK", *
FROM MyTableHank
UNION ALL
SELECT "JOHN", *
FROM MyTableJohn;
Notice I defined a field to identify the origin of the data being merged ("HANK", "JOHN"), as you suggested above.
About the forms, I believe you'll need to import them and then review the whole code. It basically depends on what the forms are doing. If they're query-based won't be a big deal (importing / fixing the queries, will make the form works). However, if the forms are related to the tables, you'll have more work to do.
Related
Good example will be shopify. Where you have N number of users (in this case each user assume site). And each user will have it's own records in DB. But db schema will be the same (same tables for each user, products, customers, orders etc.).
So question is what will be the best way to organize this kind of solution?
Store everything in one DB but in a different tables, or run separate DB for each user (but then will be question with maintaining, scalability and automatization)
possible solution:
We can use one DB with common tables like products, customers, orders etc. And we will have table users where we store records about each site.
In tables products, customers we will group all records by user_id.
This is one of possible solutions. But if we will have 1000 users (sites), each will have ~2k products, and ~100k customers, we can end up with tables which has millions of records, so questions will be:
how it will perform compare to each user (site) would have it's own DB?
how reliable this approach? bigger data, harder maintain, backup/restore
safety, if something wrong with one source thousands will be affected
Any links etc. will be much appreciated, thanks!
Create a mysql user for each tenant
Add a tenant_id column to each table
Add a view for each table that filters based on tenant_id = mysql_user
Use a trigger to automatically populate the tenant_id column on INSERT
Restrict the tenant mysql users to only access the views, not the raw tables
I wrote up a blog post on how I was able to convert a large single-tenant mysql application to a multi-tenant application in a weekend using this technique.
https://opensource.io/it/mysql-multi-tenant/
I recommend reviewing databases by well-supported open source solutions. With this in mind, here's a pretty simple schema I found real quick that'd explain a good working solution for this with scale-ability in mind.
http://www.zentut.com/sql-tutorial/sql-sample-database/
I have this file Generate_multiTanentMysql.php i do all steps with PHP script
https://github.com/ziedtuihri/SaaS_Application
Solution Design Pattern :
Creating a database user for each tenant
Renaming every table to a different and unique name (e.g. using a prefix ‘someprefix_’)
Adding a text column called ‘id_tenant’ to every table to store the name of the tenant the row belongs to
Creating a trigger for each table to automatically store the current database username to the id_tenant column before inserting a new row
Creating a view for each table with the original table name with all the columns except id_tenant. The view will only return rows where (id_tenant = current_database_username)
Only grant permission to the views (not tables) to each tenant’s database user Then, the only part of the application that needs to change is the database connection logic. When someone connects to the SaaS, the application would need to:
Connect to the database as that tenant-specific username
I'm and intern and I've been tasked with something I'm pretty unfamiliar with. My manager has requested I create a simple MySQL database using data from an Excel file(s) and I have no idea where to start. I would normally ask someone here for help but everyone seems to be really busy. Basically, the purpose of the database is to see what different object-groups relate to one another so as to keep things standardized. Trying not to go into detail about things not really relevant.
I was asked to first design a schema for the database and then I would get an update on how to implement it. Would I just start by writing queries to create tables? I'm assuming I would need to convert the Excel files to .csv, how do I read this data and send it to the correct table based on Object Type (an attribute of each object, represented in a column)?
I don't want to ask too much right now, but if someone could help me understand what I need to do to get started I would really appreciate it.
Look at the column headers in your spread sheet.
Decide which columns relate to Objects and which columns relate to Groups
The columns that relate to just Objects will become your field names for the Object table. Give this table an ID field so you can uniquely identify each Object.
The columns that relate to the Groups will become field names for a Group table. Give this table an ID field so you can uniquely identify each Group.
Think about if an Object can be in more than one Group - if so you will probably need an Object-Group table. This table would most likely contain an ObjectID and a GroupID.
I have a bit of a complex issue to deal with here at the moment and would love some help please.
I have exported data from SugarCRM in CSV format to be imported in to my new app. Now my app works slightly differently in terms of database schema. I use auto-incrementing integers for id's where as SugarCRM uses long alphanumeric strings like dc4c072c-ec0e-26b6-8780-4a9e7ec8375d.
I need to be able to take the data for say accounts and contacts, using the SugarCRM pivot table export from accounts_contacts, and change the id's, while keeping them all linked correctly. I then need to use the changed data to import in to my database, where each record in the contacts table contains an account_id field which links them to accounts, rather than a pivot table.
Now, my first thoughts would be to import all the required data in to clean database and use a lookup table with four fields in it to change the id's. So I would have old_account_id, new_account_id, old_contact_id, new_contact_id. I'd then use that table to find the right id's and amend the data.
The only issue I have with this, is that my SQL isn't amazing, so I'm having difficulty visualising and writing the query to use the lookup table and change the data. So I would love some help with this.
Also, once I have the query and the correct amended data, I need another query to then create the contacts table using an account_id field from the pivot table.
Hope you guys can help out. Thanks in advance.
Are you planning on keeping the data in sync? I'd suggest adding a "sugar_id" column in your app and a "your_app_id" column in SugarCRM. This way the native id column remains the same and that it will maintain a reference to the record in the other system.
I am creating a relational database for a friend and I have never used MS Access before. I am trying to make it so that the related values are shown on a table view rather than the numeric representation. For example, instead of
[1][2242], [2][443]
I would like
[1][John Smith], [2][Marilyn Monroe]
(the first value being the primary key and the second being the value linked to another table).
I have read this page about creating relationships between tables but I am still having trouble "viewing" the linked values. Microsoft Access is going to be the only way my friend is going to be viewing these results so that is why it is important to be able to see the correct values.
I have made relational databases before but only with traditional PHP and SQL and so I know what I need to do but I cannot seem to do it.
Any help would be appreciated.
Do not be misled into creating lookup values in tables. Create a form and add comboboxes to display the data from the related table, or use queries.
For example: create form to add records in multiple tables
Or
SELECT Id, Fullname FROM Table1
INNER JOIN Table2
ON Table1.FKID = Table2.ID
You can create the relationships visually in MS Access. You can also build queries in the query design window.
I am a bit rusty with mysql and trying to jump in again..So sorry if this is too easy of a question.
I basically created a data model that has a table called "Master" with required fields of a name and an IDcode and a then a "Details" table with a foreign key of IDcode.
Now here's where its getting tricky..I am entering:
INSERT INTO Details (Name, UpdateDate) Values (name, updateDate)
I get an error: saying IDcode on details doesn't have a default value..so I add one then it complains that Field 'Master_IDcode' doesn't have a default value
It all makes sense but I'm wondering if there's any easy way to do what I am trying to do. I want to add data into details and if no IDcode exists, I want to add an entry into the master table. The problem is I have to first add the name to the fund Master..wait for a unique ID to be generated(for IDcode) then figure that out and add it to my query when I enter the master data. As you can imagine the queries are going to probably get quite long since I have many tables.
Is there an easier way? where everytime I add something it searches by name if a foreign key exists and if not it adds it on all the tables that its linked to? Is there a standard way people do this? I can't imagine with all the complex databases out there people have not figured out a more easier way.
Sorry if this question doesn't make sense. I can add more information if needed.
p.s. this maybe a different question but I have heard of Django for python and that it helps creates queries..would it help my situation?
Thanks so much in advance :-)
(decided to expand on the comments above and put it into an answer)
I suggest creating a set of staging tables in your database (one for each data set/file).
Then use LOAD DATA INFILE (or insert the rows in batches) into those staging tables.
Make sure you drop indexes before the load, and re-create what you need after the data is loaded.
You can then make a single pass over the staging table to create the missing master records. For example, let's say that one of your staging table contains a country code that should be used as a masterID. You could add the master record by doing something along the lines of:
insert
into master_table(country_code)
select distinct s.country_code
from staging_table s
left join master_table m on(s.country_code = m.country_code)
where m.country_code is null;
Then you can proceed and insert the rows into the "real" tables, knowing that all detail rows references a valid master record.
If you need to get reference information along with the data (such as translating some code) you can do this with a simple join. Also, if you want to filter rows by some other table this is now also very easy.
insert
into real_table_x(
key
,colA
,colB
,colC
,computed_column_not_present_in_staging_table
,understandableCode
)
select x.key
,x.colA
,x.colB
,x.colC
,(x.colA + x.colB) / x.colC
,c.understandableCode
from staging_table_x x
join code_translation c on(x.strange_code = c.strange_code);
This approach is a very efficient one and it scales very nicely. Variations of the above are commonly used in the ETL part of data warehouses to load massive amounts of data.
One caveat with MySQL is that it doesn't support hash joins, which is a join mechanism very suitable to fully join two tables. MySQL uses nested loops instead, which mean that you need to index the join columns very carefully.
InnoDB tables with their clustering feature on the primary key can help to make this a bit more efficient.
One last point. When you have the staging data inside the database, it is easy to add some analysis of the data and put aside "bad" rows in a separate table. You can then inspect the data using SQL instead of wading through csv files in yuor editor.
I don't think there's one-step way to do this.
What I do is issue a
INSERT IGNORE (..) values (..)
to the master table, wich will either create the row if it doesn't exist, or do nothing, and then issue a
SELECT id FROM master where someUniqueAttribute = ..
The other option would be stored procedures/triggers, but they are still pretty new in MySQL and I doubt wether this would help performance.