database desigining for city details from multiple web service - sql-server-2008

I am working on travel application, so we have to deal with different web services like GTA, Gallileo, Kuoni etc . for getting information regarding Hotel details.
Each web service has its own list of city code and city name.
I want to design a table to store the city details from different web service, after some research I came to these two approaches
1st approach
CREATE TABLE [dbo].[City](
[CityID] [int] NOT NULL,
[CountryCode] [varchar](5) NOT NULL,
[AppCityCode] [varchar](10) NOT NULL,
[AppCityName] [varchar](200) NOT NULL,
[GTACityCode] [varchar](10) NULL,
[GTACityName] [varchar](200) NULL,
[GWSCityCode] [varchar](10) NULL,
[GWSCityName] [varchar](200) NULL,
[KuoniCityCode] [varchar](10) NULL,
....
....
....
....
....
....
)
In this approach when ever a new webservice is added then two columns (city code and city name) corresponding to the webservice is added, due to this modification and there will be a change in stored procedure and in frontend application code.
There will be no duplication while loading the cities in the textbox
2nd Approach
WSSupplier table is used to store Webservice details like GTA, Gallileo..
CREATE TABLE [dbo].[WSSupplier](
[SupplierID] [smallint] NOT NULL,
[SupplierName] [varchar](100) NOT NULL
)
CREATE TABLE [dbo].[City](
[CityID] [int] IDENTITY(1,1) NOT NULL,
[AppCityCode] [varchar](20) NULL,
[AppCityName] [varchar](150) NULL,
[CountryCode] [varchar](10) NULL,
[WSSupplierID] [smallint] NULL,
[WSCityCode] [varchar](20) NULL,
[WSCityName] [varchar](150) NULL
)
In the 2nd approach the cities will be added row by row with corresponding web service supplier ID
If new webservice come then I don't have to modify the table structure or in frontend application.
While loading cities I have to use DISTINCT to load unique city in the textbox or dropdown in frontend
In both approach I am using Appcitycode and Appcityname this will load the city textbox or dropdown in the application. While selecting the Appcityname, it will get the corresponding web service city code and send it as request to the webservice to search a hotel in a particular city.
I want to know which will be the best approach or if there is any other good approach

A third approach would be to create an intersection table between your city table and your supplier table that lists the supplier's version of the city code.
Your city table would just have your own system's city identifier. The city would appear only once. Each time you add a supplier you insert new records into the intersection table with the city codes for the cities that supplier cares about. The translation of a supplier city code to your internal city code is a simple lookup in the intersection table.
Consider something like this:
CREATE TABLE [dbo].[WSSupplier](
[SupplierID] [smallint] NOT NULL,
[SupplierName] [varchar](100) NOT NULL
)
CREATE TABLE [dbo].[City](
[CityID] [int] IDENTITY(1,1) NOT NULL,
[CityCode] [varchar](20) NULL,
[CityName] [varchar](150) NULL,
[CountryCode] [varchar](10) NULL
)
CREATE TABLE [dbo].[SupplierCityCode](
[CityID] [int] NOT NULL,
[WSSupplierID] [smallint] NULL,
[WSCityCode] [varchar](20) NULL,
[WSCityName] [varchar](150) NULL,
FOREIGN KEY [fk_city] [CityID] REFERENCES [dbo].[City],
FOREIGN KEY [fk_supplier] [WSSupplierID] REFERENCES [dbo].[WSSupplier]
)

Your question is about application and database design. From the application design point of view try to abstract from database design and think about it as some storage for your business objects. From database design point of view your question is about Database normalization - start from this article at Wikipedia as a gate to big world of database design. As for me:
CREATE TABLE [dbo].[Supplier](
[SupplierID] [smallint] NOT NULL,
[SupplierName] [varchar](100) NOT NULL
)
CREATE TABLE [dbo].[AppCity](
[CityID] [int] IDENTITY(1,1) NOT NULL,
[CityCode] [varchar](20) NULL,
[CityName] [varchar](150) NULL,
[CountryCode] [varchar](10) NULL,
)
CREATE TABLE [dbo].[SupplierCity](
[CityID] [int] IDENTITY(1,1) NOT NULL,
[SupplierID] [smallint] NOT NULL,
[CityCode] [varchar](20) NULL,
[CityName] [varchar](150) NULL
)

Related

MySQL database schema: 3 columns of same type with different purpose

I am building a financial app, and it requires the following tables:
tax
item, which can have one tax
invoice, which can have many items
To the question:
An invoice must have the ability to contain 3 categories of emails:
Recipient emails
CC emails
BCC emails
So far I just have a recipients/cc/bcc column, but I don't think it's good because I'd basically have to concatenate the emails together and separate them by a comma or something.
I also thought about a generic email table, but then I'd have to create invoice_recipient_email, invoice_cc_email and invoice_bcc_email tables respectively, in order to link the emails back to the particular invoice ID as well as categorize them by the 3 types.
Can someone advice me on my second solution, or provide a better way to do this?
Here is my current schema:
CREATE TABLE tax (
id INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
name VARCHAR(255) NOT NULL,
rate INT(3) NOT NULL,
PRIMARY KEY (id)
);
CREATE TABLE item (
id INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
name VARCHAR(255) NOT NULL,
description VARCHAR(1000) NOT NULL,
quantity INT(10) NOT NULL,
price INT(10) NOT NULL,
CONSTRAINT `f_tax_item_tax_id` FOREIGN KEY (`tax_id`) REFERENCES `tax` (`id`),
PRIMARY KEY (id)
);
CREATE TABLE invoice (
id INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
invoice_number VARCHAR(255) NOT NULL,
recipients VARCHAR(1000) NOT NULL,
cc VARCHAR(1000) NOT NULL,
bcc VARCHAR(1000) NOT NULL,
discount INT(10) NOT NULL,
note VARCHAR(500) NOT NULL,
terms VARCHAR(2000) NOT NULL,
due_date TIMESTAMP NOT NULL,
PRIMARY KEY (id)
);
I would add a single recipients table with id (pk), invoice_id (fk), email, recipient_type fields, where recipient type could have one of the following values: to, cc, or bcc. The recipient type field would tell you how to use the email address, therefore you would not need 3 separate tables to hold the 3 different recipient types.
1 record would hold only 1 email address.
You can normalize your emails into a single table and put a flag on it to categorize its type.
CREATE TABLE invoice (
id INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
invoice_number VARCHAR(255) NOT NULL,
discount INT(10) NOT NULL,
note VARCHAR(500) NOT NULL,
terms VARCHAR(2000) NOT NULL,
due_date TIMESTAMP NOT NULL,
PRIMARY KEY (id)
);
CREATE TABLE email(
id INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
invoice_id INT(10) not null references invoice(id),
email_type int not null,
addresses varchar(1000),
PRIMARY KEY (id)
);
You can define your email types in another table as well. You can look at this approach implemented in a production environment for phone numbers in MSSSQL but same concept. We however normalize the records to one phone number per record which I would also recommend you do.
CREATE TABLE [dbo].[ActorPhones](
[PKId] [int] IDENTITY(0,1) NOT NULL Primary Key,
[FKActorId] [int] NOT NULL References Actor(PKID),
[FKPhoneTypeId] [int] NOT NULL,
[Number] [varchar](20) NOT NULL
) ON [PRIMARY]
CREATE TABLE [dbo].[PhoneTypes](
[PKId] [int] IDENTITY(0,1) NOT NULL Primary Key,
[PhoneTypeName] [varchar](50) NOT NULL References ActorPhones(PKId),
[PhoneTypeDescription] [varchar](250) NULL,
[ModifiedDate] [datetime] NULL,
[ModifiedBy] [varchar](50) NULL
)

SSIS migration: split one record to many tables

I'm refactoring my db's User object from a schema that combines BillingAddress with Shipping Address:
[BillingFirstName] [nvarchar](50) NOT NULL,
[BillinglastName] [nvarchar](50) NOT NULL,
[BillingAddress] [nvarchar](100) NOT NULL,
[BillingCity] [nvarchar](100) NOT NULL,
[BillingZip] [varchar](16) NOT NULL,
[BillingState] [nvarchar](2) NOT NULL,
[shippingFirstName] [nvarchar](50) NULL,
[shippingLastName] [nvarchar](50) NULL,
[shippingAddress] [nvarchar](100) NULL,
[shippingCity] [nvarchar](100) NULL,
[shippingState] [nvarchar](2) NULL,
[shippingZip] [nvarchar](20) NULL,
[shippingPhone] [nvarchar](30) NULL,
Refactored to one table for User and a separate table for addresses bound by a foreign key Users.ID => Addresses.idUser
CREATE TABLE [dbo].[Addresses](
[Id] [int] IDENTITY(1,1) NOT NULL,
[Type] [nchar](10) NOT NULL, // designates Billing or Shipping
[Formatted] [nchar](600) NOT NULL,
[Street] [nchar](100) NOT NULL,
[City] [nchar](100) NOT NULL,
[POBox] [nchar](50) NULL,
[Region] [nchar](50) NULL,
[PostalCode] [nchar](50) NULL,
[Country] [nchar](50) NULL,
[ExtendedAddress] [nchar](100) NULL,
[idUser] [int] NULL,
How do I tell SSIS to import a record to the simplified User object and then create 2 addresses records; one with the Shipping info and the other with Billing?
I'd want preserve existing ID Key.
thx
Multicast your source data.
Add a Derived Column component to each output stream. In your mind, designate one as the "Billing Address" stream and one as the "Shipping Address" stream.
Add a new column named "Type", hardcoded to "Billing" and "Shipping", respectively.
Add a destination component to each streams both pointing to your Address table.
Map the appropriate columns in each stream (ie BillingCity to the City in the "billing address" stream, ShippingCountry to the Country in the "shipping address" stream, etc.)

mysql queries to add simple prodcts in magento

Any one can help me to insert 1000 products of one database table into magento
I have one database table of product i want to insert all products of this table into magento store using mysql queries
Database Table Structure
CREATE TABLE IF NOT EXISTS `product` (
`id` int(11) NOT NULL,
`title` text NOT NULL,
`author` text NOT NULL,
`isbn13` bigint(13) NOT NULL,
`isbn10` varchar(10) NOT NULL,
`rank` varchar(10) NOT NULL,
`pic` varchar(25) NOT NULL,
`price` int(10) NOT NULL,
`publicationdate` varchar(10) NOT NULL,
`publisher` varchar(255) NOT NULL,
`desc1` text NOT NULL,
`cat` varchar(25) NOT NULL
)
To ensure that your data goes in correctly, you probably should use the Magento import feature , the official tutorial is available here.
If you attempt to do it via a direct mysql insert you may end up missing some of the EAV data that Magento requires...
If you need something a little faster, then you can use Magmi to import the data.
I am using Magmi on a daily basis to import/update around 3500 configurable and simple products without any issues.

SQL Server 2008 - Too much denormalization and over Indexing: What use is there for the Matrix?

I have a budding developer who is very enthusiastic about something he is calling “the matrix”
I am looking for peer insight
In a nutshell this is what we have:
- 1 highly denormalized table with about 120 columns
- Data points range from account, customer, household, relationship, product, employee, etc…
- One index per column: about 120 non-clustered indexes
- About 90% of all space in the database used by indexes today are indexes on this table
- Today about 1.5 million rows with a lot of nulls
- Table loaded with a stored procedure whose core is dynamic SQL
- All Field names are generic and do not describe the data
- A data dictionary type table is used with the dynamic SQL to load any data point to any field
- Field mapping is not static: today column dim_0001 is customer name, but tomorrow maybe something else
- No primary key
- No foreign keys
- No real constraints (For example all fields are nullable)
The argument for the table:
- Makes writing queries simpler because it eliminates the needs to write some join
The intended use:
- An End User Layer and would be a core component of a Universe build in Business Objects
- Post ETL process development
My recommendation will either kill the process where it is today (early development in a test environment) or move it to the next step in test.
Based on the research I have done, my education, and experience I do not support it and want the tables dropped as soon as the one or two processes that depend on these tables have been migrated to another solution.
Script below for your reference (I limited to one index example).
Any insight you can offer (even just a one word opinion) is valuable
-- The Matrix
CREATE TABLE [z005497].[tblMatrix](
[as_of_dt] [datetime] NOT NULL,
[dim_0001] [varchar](100) NULL,
[dim_0002] [varchar](103) NULL,
[dim_0003] [varchar](100) NULL,
[dim_0004] [varchar](100) NULL,
[dim_0005] [varchar](100) NULL,
[dim_0006] [varchar](100) NULL,
[dim_0007] [varchar](100) NULL,
[dim_0008] [varchar](100) NULL,
[dim_0009] [varchar](100) NULL,
[dim_0010] [varchar](100) NULL,
[dim_0011] [varchar](100) NULL,
[dim_0012] [varchar](100) NULL,
[dim_0013] [varchar](100) NULL,
[dim_0014] [varchar](100) NULL,
[dim_0015] [varchar](100) NULL,
[dim_0016] [varchar](100) NULL,
[dim_0017] [varchar](103) NULL,
[dim_0018] [varchar](103) NULL,
[dim_0019] [varchar](103) NULL,
[dim_0020] [varchar](103) NULL,
[dim_0021] [varchar](103) NULL,
[dim_0022] [varchar](103) NULL,
[dim_0023] [varchar](103) NULL,
[dim_0024] [varchar](103) NULL,
[dim_0025] [varchar](103) NULL,
[dim_0026] [varchar](11) NULL,
[dim_0027] [varchar](11) NULL,
[dim_0028] [varchar](11) NULL,
[dim_0029] [varchar](11) NULL,
[dim_0030] [varchar](11) NULL,
[dim_0031] [varchar](11) NULL,
[dim_0032] [varchar](11) NULL,
[dim_0033] [varchar](11) NULL,
[dim_0034] [varchar](11) NULL,
[dim_0035] [varchar](11) NULL,
[dim_0036] [varchar](11) NULL,
[dim_0037] [varchar](11) NULL,
[dim_0038] [varchar](11) NULL,
[dim_0039] [varchar](11) NULL,
[dim_0040] [varchar](11) NULL,
[dim_0041] [varchar](11) NULL,
[dim_0042] [varchar](11) NULL,
[dim_0043] [varchar](11) NULL,
[dim_0044] [varchar](11) NULL,
[dim_0045] [varchar](11) NULL,
[dim_0046] [varchar](11) NULL,
[dim_0047] [varchar](11) NULL,
[dim_0048] [varchar](11) NULL,
[dim_0049] [varchar](11) NULL,
[dim_0050] [varchar](11) NULL,
[dim_0051] [varchar](11) NULL,
[dim_0052] [varchar](11) NULL,
[dim_0053] [varchar](11) NULL,
[dim_0054] [varchar](5) NULL,
[dim_0055] [varchar](5) NULL,
[dim_0056] [varchar](5) NULL,
[dim_0057] [varchar](5) NULL,
[dim_0058] [varchar](5) NULL,
[dim_0059] [varchar](5) NULL,
[dim_0060] [varchar](5) NULL,
[dim_0061] [varchar](5) NULL,
[dim_0062] [varchar](5) NULL,
[dim_0063] [varchar](5) NULL,
[dim_0064] [varchar](5) NULL,
[dim_0065] [varchar](5) NULL,
[dim_0066] [varchar](5) NULL,
[dim_0067] [varchar](5) NULL,
[dim_0068] [varchar](5) NULL,
[dim_0069] [varchar](5) NULL,
[dim_0070] [varchar](5) NULL,
[dim_0071] [varchar](5) NULL,
[dim_0072] [varchar](5) NULL,
[dim_0073] [varchar](5) NULL,
[dim_0074] [varchar](5) NULL,
[dim_0075] [varchar](5) NULL,
[dim_0076] [varchar](5) NULL,
[dim_0077] [varchar](5) NULL,
[dim_0078] [varchar](5) NULL,
[dim_0079] [varchar](5) NULL,
[dim_0080] [varchar](5) NULL,
[dim_0081] [varchar](5) NULL,
[dim_0082] [varchar](5) NULL,
[dim_0083] [varchar](5) NULL,
[dim_0084] [int] NULL,
[dim_0085] [int] NULL,
[dim_0086] [int] NULL,
[dim_0087] [int] NULL,
[dim_0088] [int] NULL,
[dim_0089] [int] NULL,
[dim_0090] [int] NULL,
[dim_0091] [int] NULL,
[dim_0092] [int] NULL,
[dim_0093] [int] NULL,
[dim_0094] [varchar](12) NULL,
[dim_0095] [varchar](12) NULL,
[dim_0096] [varchar](12) NULL,
[dim_0097] [varchar](120) NULL,
[dim_0098] [varchar](120) NULL,
[dim_0099] [varchar](120) NULL,
[dim_0100] [numeric](20, 0) NULL,
[dim_0101] [varchar](20) NULL,
[dim_0102] [varchar](20) NULL,
[dim_0103] [varchar](20) NULL,
[dim_0104] [varchar](20) NULL,
[dim_0105] [varchar](20) NULL,
[dim_0106] [varchar](20) NULL,
[dim_0107] [varchar](20) NULL,
[dim_0108] [varchar](20) NULL,
[dim_0109] [varchar](20) NULL,
[dim_0110] [varchar](20) NULL,
[dim_0111] [varchar](20) NULL,
[dim_0112] [varchar](20) NULL,
[dim_0113] [varchar](20) NULL,
[dim_0114] [varchar](20) NULL,
[dim_0115] [varchar](20) NULL,
[dim_0116] [varchar](20) NULL,
[dim_0117] [varchar](20) NULL,
[dim_0118] [varchar](20) NULL,
[dim_0119] [varchar](20) NULL,
[dim_0120] [varchar](20) NULL,
[lastLoad] [datetime] NULL
) ON [PRIMARY]
-- Index example
CREATE NONCLUSTERED INDEX [idx_dim_0001 (not unique)] ON [z005497].[tblMatrix]
(
[dim_0001] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
-- The configuration table from which developers would find out what is in the Matrix
CREATE TABLE [z005497].[tblMatrixCfg](
[dimId] [int] IDENTITY(100000,1) NOT NULL,
[colName] [varchar](25) NOT NULL,
[dataType] [varchar](25) NOT NULL,
[dimName] [varchar](25) NOT NULL,
[dimDesc] [varchar](500) NOT NULL,
[dimpath] [varchar](5000) NOT NULL,
[loadDate] [datetime] NOT NULL,
[modUser] [varchar](100) NOT NULL,
[modDate] [datetime] NOT NULL,
CONSTRAINT [PK_tblMatrixCfg_1] PRIMARY KEY CLUSTERED
(
[dimId] ASC,
[colName] ASC,
[dimName] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
Kill it if you can.
Also, that developer needs a lot more experience. And he/she should get it at another company.
It's basically violating so many things I don't know where to start.
Even if you end up fighting a highly normalized model which is following someone's best practices slavishly, it won't compare to the disaster which this design is going to create.
Just to give one example of what Cade meant with "I don't know where to start" :
"today column dim_0001 is customer name, but tomorrow maybe something else"
This typically also means that in the User acceptance system, dim_0001 can be customer name (and the system might seem to work and get accepted), and then you move to production, and dim_0001 gets to be name of the president's wife or so, and then hours of meetings need to be spent trying to figure out (a) where the problem is, and (b) how to get it fixed in as little time as possible.
( (b) usually amounts to patching the code with stuff like "if col_name = dim_0001 then don't treat it as what the matrix says it is, but treat it as what is hardcoded here instead".)
"What use is there for the Matrix?"
Well, I certainly don't get it.
I have never seen anything like this before and I don't understand how it is meant to be used or how the indexes is meant to speed up anything or how it is possible to query this table without using at least self joins.
Call me inexperienced if you like but this is a first for me. I would think that if this is the way to do things, the db vendors should not put so much effort into allowing us developers to define tables, with columns that have different data types, with relationships.
This is the result of trying to stuff an object oriented paradigm into a relational system. Document databases allow for this sort of programming:
Documents inside a document-oriented database are similar, in some
ways, to records or rows, in relational databases, but they are less
rigid. They are not required to adhere to a standard schema nor will
they have all the same sections, slots, parts, keys, or the like. For
example here's a document:
FirstName="Bob", Address="5 Oak St.", Hobby="sailing".
Another document could be:
FirstName="Jonathan", Address="15 Wanamassa Point Road", Children=[{Name:"Michael",Age:10}, {Name:"Jennifer", Age:8},
{Name:"Samantha", Age:5}, {Name:"Elena", Age:2}].
Both documents have some similar information and some different.
Unlike a relational database where each record would have the same set
of fields and unused fields might be kept empty, there are no empty
'fields' in either document (record) in this case. This system allows
new information to be added and it doesn't require explicitly stating
if other pieces of information are left out.
Trying to use this paradigm in a relational database is a "square peg, round hole" problem. A document database might be excellent for a highly transactional system, but analysis would be better served by loading the transactional data into various fact tables in a data warehouse.

Use hierarchyid to store address of a customer

I have a table named 'AddressDemo' to store address of a customer with the following fields,
CREATE TABLE [dbo].[AddressDemo](
[AddressID] [int] IDENTITY(1,1) NOT NULL,
[State] [nvarchar](50) NULL,
[District] [nvarchar](50) NULL,
[Taluk] [nvarchar](50) NULL,
[Village] [nvarchar](50) NULL,
[Street1] [nvarchar](50) NULL,
[Street2] [nvarchar](50) NULL,
[Phone] [nvarchar](50) NULL,
[Mobile] [nvarchar](50) NULL,
[Email] [nvarchar](50) NULL,
CONSTRAINT [PK_AddressDemo] PRIMARY KEY CLUSTERED
(
[AddressID] ASC
))
Where there is a hierarchy exists, which is akin to
State --> District --> Taluk --> Village --> Street1 --> Street2
Isn't it a good idea to keep a separate table to store the hierarchy so that we can avoid duplication of data. How is the following
CREATE TABLE [dbo].[LocationDemo](
[LocationID] [int] IDENTITY(1,1) NOT NULL,
[LocationNodeID] [hierarchyid] NULL,
[Location] [nvarchar](50) NULL,
CONSTRAINT [PK_LocationDemo] PRIMARY KEY CLUSTERED
(
[LocationID] ASC
))
So the 'AddressDemo' will look like the following
CREATE TABLE [dbo].[AddressDemo](
[AddressID] [int] IDENTITY(1,1) NOT NULL,
[LocationID] [int] NULL,
[Phone] [nvarchar](50) NULL,
[Mobile] [nvarchar](50) NULL,
[Email] [nvarchar](50) NULL,
CONSTRAINT [PK_AddressDemo] PRIMARY KEY CLUSTERED
(
[AddressID] ASC
))
and LocationID of AddressDemo reference to LocationID of LocationDemo.
While your proposed solution is more dynamic than the flattened solution you described I would not go with a completely dynamic schema for locations in this case. Adding hierarchical processing is not something to be done without good reason because it complicates your database queries later on and limits your performance optimisation alternatives (views containing CTEs cannot be indexed, and you would need views to reasonably consume this data by your application).
If you're talking about a low volume system or one in which the number of addresses being stored is small you can play with the dynamic address element route, but considering the fact that no one address would logically exist without the majority of the location elements I would again say it's overkill.
Go for a more normalized route without going overboard. Consider making a State table and a FK to that table from Address, a District table and a FK and so on...