Table Image.
I want to distribute the data of above table into multiple tables.
Say :-
Product name and Company goes into 1st table.
Barcode and Price goes into 2nd table.
Category and Subcategory goes into 3rdtable.
One approach to solve your problem would be to implement a custom management command, you can check the documentation here. You could parse the CSV and then update the specific entries on your database.
The usage would be something like this (assuming for example that the command is called updateproducts):
$ python manage.py updateproducts path/to/your/file.csv
Of course that depending on the size of your data, other approaches might be more efficient.
Related
I intend to clean some data in a MySQL database table for some data that was not properly captured. The table in question contains numerous columns but the column of interest is named id_number. Sample data in the column is captured as follows: BUT3345622019, PAC1034412016, etc. in that format. However, my requirement is to re-write each value in the column replacing it with values like BUT-3-3456-2/2019, PAC-1-0344-1/2016, etc. I have been thinking about the perfect SQL script (in MySQL) to solve the problem but so far I have not been successful. The table name is users. I am requesting for some help on this particular problem.
I asked a question a few days ago to know how to import an existing database into Neo4J. Thanks to the person who explained me how to do that. I decided to create a CSV file from my database (around 1 million entries) and to load it from the Neo4j webadmin to test it. The problem is that each row of this database contains redundant data, for example my database contains actions from different users but each user can do mutliple actions. The structure of my graph would be to create a node for each user that is linked to each action he does. That's why I have to create only one node for each user even if his name appears in several rows of my CSV file (because he made several actions). What is the method to do that ? I guess it's possible to do that in Cypher right ?
Thanks a lot
Regards
Sam
In case you have references that might or might not exist, you should use the MERGE statement. MERGE either finds something or creates something in your database.
Please refer to the respective section in the reference manual: http://docs.neo4j.org/chunked/stable/cypherdoc-importing-csv-files-with-cypher.html. Here the country is shared my multiple users there the country is merged wheres the users and their relationships to countries are unconditionally created.
I have a bit of a complex issue to deal with here at the moment and would love some help please.
I have exported data from SugarCRM in CSV format to be imported in to my new app. Now my app works slightly differently in terms of database schema. I use auto-incrementing integers for id's where as SugarCRM uses long alphanumeric strings like dc4c072c-ec0e-26b6-8780-4a9e7ec8375d.
I need to be able to take the data for say accounts and contacts, using the SugarCRM pivot table export from accounts_contacts, and change the id's, while keeping them all linked correctly. I then need to use the changed data to import in to my database, where each record in the contacts table contains an account_id field which links them to accounts, rather than a pivot table.
Now, my first thoughts would be to import all the required data in to clean database and use a lookup table with four fields in it to change the id's. So I would have old_account_id, new_account_id, old_contact_id, new_contact_id. I'd then use that table to find the right id's and amend the data.
The only issue I have with this, is that my SQL isn't amazing, so I'm having difficulty visualising and writing the query to use the lookup table and change the data. So I would love some help with this.
Also, once I have the query and the correct amended data, I need another query to then create the contacts table using an account_id field from the pivot table.
Hope you guys can help out. Thanks in advance.
Are you planning on keeping the data in sync? I'd suggest adding a "sugar_id" column in your app and a "your_app_id" column in SugarCRM. This way the native id column remains the same and that it will maintain a reference to the record in the other system.
I am in need of a solution.
I am supposed to load the data of a table from PROD server to UAT. If records are missing in UAT, load the missing rows. How Should i go about it ?
Second Problem.
I am fetching some data (EmpId,NAME,CreditCardNumebr) from some text files. They are collaborated based on EmpId from a table in SQL Server (ID,Address,ContactNumber).
The combined information (ID,NAME,ContactNumber,Address,Creditcard) have to be loaded in the main table. IF the record doesn't exist, ADD. But if some information is missing in the fields of the records present, UPDATE.
I was able to get some information from Lookup Video session uploaded.
But not able to do the required things.
Please help.
To join the data of your two sources you should use a "merge join" component or a "Lookup" component. It depends how many rows you've in both sources. Once your two sources have been joined you should write this result in a staging table. Then apply a sql merge statement between the staging and the final destination tables.
Probably not what you are looking for but if it is incremental loads, you can import the data to a "Stage" table and write a query to do a update insert into the active tables. Let it compare the Primary keys. If it is the same, test the fields for changes and update, if not, insert new row.
Hope it help.
I can't have a Staging table. That is a requirement.
Anyway I did make a partial solution for the problem.
We need to use 2 LookUp Transformations to get the desired result.
1 For collabarating data of the Flat file and the table that holds the partial data.
1 For checking for record existance based on the business key (i.e. ID (Primary Key))
Flat File Source --> LookUp (For collabaration) --> LookUp (For record check) --> OleDb Destination
The records that comes out in the (NO Match Output) are filled in the table.
I need to find out the way to update the records (Which come in the Match output)
If you guys can provide me a solution for it , it will be highly appreciated.
I'm developing website in which will be categorized advertisements. In each category will be possible different fields of input (example: for car there will be motor size, for cat there will be a race). So I'm thinking how to build database to manage this (I will use MYSQL database). One way you can see in attached picture, I know that also is solution to create table for each values datatape, but I'm wondering that it will slow down a website. This solution which is in picture will generate empty fields in sp_advertisement_value table what isn't good also.
What is in your opinion the best solution? Maybe there is something else?
p.s. Here is a link to database market.
You can store it like name/value pairs (more or less same to what you is described in the image you attached).
A simple schema would be a table having two columns name and value. Instead of having a column for each data type like value_int, value_string etc. have one single column value who's data type can be varchar (or Text as seems fit to you). You can do all the data conversion in your application code as per your needs.
You can do some normalization here too for instance instead of saving name you can make a separate lookup table named parameters having id, name and other related information and have the parameter_id in the table where you are storing parameter values.