How to merge two tables into one using mongify - mysql

I am planning to migrate my application database from Mysql to Mongo with low schema changes. In my new schema I merged two Mysql tables into one Mongo collection. I want to use mongify (https://github.com/anlek/mongify) gem to populate my existing Mysql data into Mongo with newer schema.
How to do this? Is there a way in mongify to merge two Mysql tables into one?
Mysql tables
user
id
name
nickname
type
user_role
id
role
alias
user_id
I merged the above two tables into single collection in Mongo
user
id
name
type
role
alias

Try Left Join: (Tables will be merged)
SELECT column_name(s)
FROM table1
LEFT JOIN table2
ON table1.column_name=table2.column_name;
Export that data to sql format and upload it to mongodb
SELECT country INTO customerCountry
FROM customers
WHERE customerNumber = p_customerNumber;
CASE customerCountry -- Here you have to see if that alias data is set
WHEN 'USA' THEN
SET p_shiping = '2-day Shipping'; -- here you have to write Update Query
ELSE
SET p_shiping = '5-day Shipping';
END CASE;
I think it might help you

Fetch the data using JOIN from MySQL and load that data into MongoDB:
Try this:
SELECT U.id, U.name, U.type, UR.role
FROM USER U
INNER JOIN user_role UR ON U.id = UR.user_id;

An alternative if its difficult to merge two tables ad-hoc then run a mongo shell script post-hoc; they are very easy to program (couple o' lines of js) and simple to execute. plus one can apply any needed conversion, such as into json/bson arrays permitted in mongo but not the mysql.
assuming a user and user_role collection (or table) in mongo.
db.user_role.find({}).forEach(function(doc) {
var roles = doc.roles;
// convert mysql format here if needed.
//var roles_array = JSON.parse(roles) // or whatever?
//var alias = doc.alias;
// if alias is null empty or ??
// dont upsert to avoid orphaned user_role records
db.user.update({"_id": doc.user_id}, {"roles": roles, "alias", doc.alias }, {upsert: false, multi: false});
}
Then execute using the mongo shell
mongo localhost:27017/test myjsfile.js

Related

How to update multiple table at once using TypeORM

I'm using TypeORM with NestJS and I want to do this query to a MySQL DB:
UPDATE user u, table2 t2
SET u.is_active = 0, t2.is_active = 0
WHERE u.id = 'id'
AND t2.user_id = u.id;
Relations are:
User:
#OneToMany(
() => Table2,
(t2) => t2.user,
{
onDelete: 'CASCADE',
eager: true
}
)
field_table_2: Table2[]
And Table2
#ManyToOne(
() => User,
(user) => user.field_table_2
)
#JoinColumn({name:'user_id'})
user: User
The point is: One user can have many table2 but every row in table2 depends for only one user. Then a relation one-to-many and many-to-one. All right so far.
When I delete one user, I don't want remove the data into DB, I want to set field is_active to false in both tables.
I can do it in a single query using SQL but using TypeORM I've only found a solution using raw sql:
async remove(id: string) {
return await this.dataSource.query(`
UPDATE user u, table2 t2
SET u.is_active = 0, t2.is_active = 0
WHERE u.id = ?
AND t2.user_id = u.id`,[id])
}
But having an ORM I don't like to use raw SQL... how can I get the query I want using the ORM?
I know I can create a transaction, update first one table, then the other table and if everything is ok commit the transaction.
But, if I can do it in a single query, is efficient to do it in a transaction? I refer also for simplicity in a single query.
Thanks in advance.
TypeORM does not currently have an expressive way of updating multiple tables in a single query this via either query builder or repository methods. Currently those are limited to updating a single table.
I would say in your case, if you wish to do this in a single query pull out isActive into an entity / table of its own. Then both user and table 2 could have a 1-1 relationship with this new entity. Therefore, you only need to update one table entry.
Otherwise, this can be done with two separate update queries and a transaction. How you will perform the update is more a matter of choice, could use Repository or QueryBuilder.
There are also several options for managing transactions, the most fine grained of which is QueryRunner.
Pseudocode might look something like this:
queryRunner.startTransaction();
try {
await queryRunner
.createQueryBuilder()
.update(EntityA)
.set({isActive: false})
.execute();
await queryRunner
.createQueryBuilder()
.update(EntityB)
.set({isActive: false})
.execute();
await queryRunner.commitTransaction();
}
catch (e) {
queryRunner.rollbackTransaction()
}
However, I suspect that this could all also be done using CASCADE and softDelete

SELECT * FROM db1 WHERE db1.table.value = db2.table.value

I'm working with mySQL db and trying to display the correct data for the user. In order to do that I check if the data that I call from one backend is equal to username from another backend like so
SELECT * FROM db1 WHERE db1.table.value = db2.table.value
Names of databases are A and B.
SELECT *
FROM `A.onboardings`
, `B.loginsystem`
WHERE onboardings.sales_email = loginsystem.username
The problem is I get an error A.A.onboardings doesn't exists and A.B.loginsystem doesn't exist pls help :(
You must use this form - from A onboardings
You have to put the backticks in the right pace, or else mysql things your table is called A.onboardings
As seen bleow the needs to be around the database and the table name
And the use of aliases helps to keep even in big queries a good overview and yu have to write less
"SELECT * FROM `A`.`onboardings` a1,`B`.`loginsystem` b1 WHERE a1.sales_email = b1.username"
Try this one( Change the query according to your DB name, table, and matching column name)
SELECT * FROM mydatabase1.tblUsers INNER JOIN mydatabase2.tblUsers ON mydatabase1.tblUsers.UserID = mydatabase2.tblUsers.UserID
The problem is that
`A.onboardings`
is not the same as
A.onboardings
The first is a table reference where there table name has a period in it. The second is for the onboardings table in database A.
In addition, you should be using JOIN!!!
SELECT *
FROM A.onboardings o JOIN
B.loginsystem ls
ON o.sales_email = ls.username;
If you feel compelled to escape the identifies -- which I do not recommend -- then:
SELECT *
FROM `A`.`onboardings` o JOIN
`B`.`loginsystem` ls
ON o.sales_email = ls.username;

MySQL can only use 61 tables in a join; How can I bypass this error for the following query?

I get a MySQL Error saying, I cannot use more than 61 tables in a join. I need to avoid this error. How do I do it? Please Help.
select
view_pdg_institutes.user_id as User_ID,
view_pdg_institutes.institute_id as Teacher_ID,
view_pdg_institutes.institute_name as Institute_Name,
view_pdg_institutes.user_email as Email,
view_pdg_institutes.contact_person_name as Contact_Person,
view_pdg_institutes.alternative_contact_no as Alternative_Mobile_No,
view_pdg_institutes.primary_contact_no as Mobile_No,
view_pdg_institutes.correspondance_address as Address,
view_pdg_institutes.other_communication_mode as Preferred_Contact_Mode,
view_pdg_institutes.size_of_faculty as Size_of_Faculty,
view_pdg_institutes.operation_hours_from as Operation_Hours_From,
view_pdg_institutes.operation_hours_to as Operation_Hours_To,
view_pdg_institutes.teaching_xp as Teaching_Experience,
view_pdg_institutes.installment_allowed as Installment_Allowed,
view_pdg_institutes.about_fees_structure as About_Fees_Structure,
view_pdg_institutes.no_of_demo_class as No_of_Demo_Classes,
view_pdg_institutes.demo_allowed as Demo_Allowed,
view_pdg_institutes.price_per_demo_class as Price_Per_Demo_Class,
view_pdg_tuition_batch.tuition_batch_id as Batch_ID,
view_pdg_batch_subject.subject_name as Subject_Name,
view_pdg_batch_subject.subject_type as Subject_Type,
view_pdg_batch_subject.academic_board as Academic_Board,
view_pdg_batch_fees.fees_type as Fees_Type,
view_pdg_batch_fees.fees_amount as Fees_Amount,
view_pdg_tuition_batch.course_days as Course_Days,
view_pdg_tuition_batch.days_per_week as Days_Per_Week,
view_pdg_tuition_batch.class_duration as Class_Duration,
view_pdg_tuition_batch.class_type as Class_Type,
view_pdg_tuition_batch.course_length as Course_Length,
view_pdg_tuition_batch.course_length_type as Course_Length_Type,
view_pdg_tuition_batch.no_of_locations as No_of_Locations,
view_pdg_tuition_batch.class_capacity_id as Class_Capacity_ID,
view_pdg_tutor_location.locality as Locality,
view_pdg_tutor_location.address as Address,
view_pdg_batch_class_timing.class_timing as Class_Timing
from view_pdg_tuition_batch
left join view_pdg_institutes on (view_pdg_tuition_batch.tutor_institute_user_id = view_pdg_institutes.user_id)
left join view_pdg_batch_subject on (view_pdg_batch_subject.tuition_batch_id = view_pdg_tuition_batch.tuition_batch_id)
left join view_pdg_batch_fees on (view_pdg_batch_fees.tuition_batch_id = view_pdg_tuition_batch.tuition_batch_id)
left join view_pdg_batch_class_timing on (view_pdg_batch_class_timing.tuition_batch_id = view_pdg_tuition_batch.tuition_batch_id)
left join view_pdg_tutor_location on (view_pdg_tutor_location.tuition_batch_id = view_pdg_tuition_batch.tuition_batch_id)
group by view_pdg_tuition_batch.tuition_batch_id;
I need a solution that would not require changing the current approach of writing the query.
I don't think it's possible to do what you're asking without some elaborate changes in the way you store and query data. You can
denormalize your DB to store JSON data;
create materialized views, emulating them via triggers, because they're absent in MySQL;
use temporary tables;
join partial selects by hand at the call site;
compile MySQL with another join limit;
use proper SQL engine like Postgres, that doesn't suffer from such stupid things.
Insert the contents of each view into its own temporary table. Then do the same query with the temporary table names substituted for the original view names.

Accidently deleted all data in the 'master' database

I've used this script by accident on the 'master' database instead of a temp database.
sp_msforeachtable 'delete from ?'
Has it caused any harm? If so, how can I restore the data?
No it shouldn't have deleted anything (assuming you have no user tables in master).
Testing
exec sys.sp_MSforeachtable 'select ''?'''
Doesn't return anything for me. So it seems to exclude the system tables such as spt_values.
Edit: Indeed Looking at the definition of the procedure it does only include tables where OBJECTPROPERTY(o.id, N'IsUserTable') = 1
Martin Smith is right to say that sp_MSforeachtable does not delete system tables.
However, though we may think of tables such as spt_values and MSreplication_options as system tables, they are in fact user tables according to SQL Server.
When I run this query in my master database:
SELECT name, OBJECTPROPERTY(object_id, N'IsUserTable') AS IsUserTable
FROM master.sys.tables;
I see the following result set:
name IsUserTable
--------------------- -----------
spt_fallback_db 1
spt_fallback_dev 1
spt_fallback_usg 1
spt_monitor 1
MSreplication_options 1
So how was Stijn saved from a reinstall?
If you look at how sp_MSforeachtable is implemented, you will see it does something like this to select the tables to drop:
declare #mscat nvarchar(12)
select #mscat = ltrim(str(convert(int, 0x0002)))
SELECT *
from dbo.sysobjects o join sys.all_objects syso on o.id = syso.object_id
where OBJECTPROPERTY(o.id, N'IsUserTable') = 1 and o.category & #mscat = 0;
In my master database, this returns an empty result set.
The where clause applies a bitmask to the category column of table sysobjects to exclude tables that are not 'mscat'.
So the tables in the master database are protected not because they are system tables, but because they are 'Microsoft' tables.
This use of the category column is completely undocumented in Books Online All it has is a vague description:
Used for publication, constraints, and identity.
But the sysobjects table is deprecated anyway, so you shouldn't be using it. :)
An equivalent query using the supported view sys.tables would look like this:
SELECT *
FROM sys.tables
WHERE is_ms_shipped = 0;
In my master database, this also returns an empty result set.

Store procedure select all fields from One table using join

I am very frustrated from linq to sql when dealing with many to many relationship with the skip extension. It doesn't allow me to use joinned queries. Not sure it is the case for SQL server 2005 but I am currently using SQL Server 2000.
Now I consider to write a store procedure to fetch a table that is matched by two tables e.g. Album_Photo (Album->Album_Photo<-Photo) and Photo table and only want the Photos data so I match the Album's ID with Album_Photo and use that ID to match the photo. In the store procedure I am just fetch all the joinned data. After that in the linq to sql, I create a new Album object.
e.g.
var albums = (from r in result
where (modifier_id == r.ModifierID || user_id == r.UserID)
select new Album() {
Name = r.Name,
UserID = r.UserID,
ModifierID = r.ModifierID,
ID = r.ID,
DateCreated = r.DateCreated,
Description = r.Description,
Filename = r.Filename
}).AsQueryable();
I used the AsQueryable to get the result as a IQueryable rather than IEnumerable. Later I want to do something with the collection, it gives me this error:
System.InvalidOperationException: The query results cannot be enumerated more than once.
It sounds like you have a situation where the query has already executed by the time you are want to filter it later in your code.
Can you do something like...
var albums = (blah blah blah).AsQueryable().Where(filterClause) when you have enough info to process
what happens if you try albums.where(filter) later on in the code? Is this what you are trying?