I have a question regarding the correct design for Google Engine & Google Maps.
Requirement:
Have a list of stores with their individual attributes (location, hours, etc.).
Each store can be mapped to multiple store types (outlet, retail, business, etc.).
I understand that #1 can be achieved by importing the my data into my Google Maps Engine in a table. #2 is a different set of data with Store PKID as a foreign key and another column specifying the store type. #2 does not have a PKID - i.e. 1 store FK can map to multiple store types and hence, will be represented by multiple rows in #2.
Example:
Table 1:
Store ID, Hours
1,5-8
2,5-5
3,5-5
Table 2:
Store ID, Type
1, Outlet
1, Retail
2, Business
3, Retail
Can I query the two tables in one go using Google Maps API?
OR
Do I have to first get the list of stores from table 1 and then query table 2 for each store?
What design considerations should I keep in mind?
Thank you!
Answer my own question for the benefit of others:
From an application perspective, the number of external calls should be as minimal as possible. There is a fine line between ideal system architecture and performance/costs of the architecture.
In the above example, it would require two calls to get the job done. In addition, the 2nd table is not being used for anything else; hence, I have decided to implement the below table structure:
Table 1:
Store ID, Hours, Outlet, Retail, Business
1,5-8,true,false,false
2,5-8,false,true,true
Now, I do realize that the data is not normalized; however, I have reduced # of external queries.
Related
How do I use Power Pivot to summarize data in groups which are defined in a separate, non-relatable table
I'm analyzing a database that has the following tables:
Sales
Store
Category
Units
Sales
Stores
Store
address
etc
StoreGroups
Store
Group
A store can be in multiple groups (i.e. store B762 is in NW group & control_group) hence the StoreGroup table - where the two fields together make the primary key. Therefore, I can't relate StoreGroups to my Sales table, because both have duplicate Store values.
Right now all stores are being reported in each group:
PivotTableScreenshot
to confirm, if a store is in two groups, its sales should get counted for BOTH groups i.e. control group and NWRegion.
I've tried to adapt this DAX example mentioned below but have not been successful:
http://www.daxpatterns.com/dynamic-segmentation/
You have a many-to-many relationship between stores and groups.
You should be able to create a relationship from the store in StoreGroups and the Store in Stores (StoreGroups is a bridge table).
If you can post a link to some sample data, that would be helpful.
After doing that, you can read start to read about writing DAX formulas for many-to-many scenarios here. Be sure to also read in the comments, especially the one from Marco Russo.
I’ve been thinking about this for a couple of days but I feel that I’m lacking the right words in order to ask google the questions I need an answer to. That’s why I’d really appreciate an kind of help, hints or guidance.
First of all, I have almost no experience with databases (apart from misusing Excel as such) and, unfortunately, I have all my data written in very impractical and huge .csv files.
What I have:
I have time series data (in 15 minute-steps) for several hundred sensors (SP) over the course of several years (a couple of million rows in total) in Table 1. There are also some weather condition data (WCD) that applies to all of my sensors and is therefore stored in the same table.
Note that each sensor delivers two data points per measurement.
Table1 (Sensors as Columns)
Now I also have another table (Table 2) that lists several static properties that define each sensor in Table 1.
Table 2 (Sensors as Rows)
My main question is concerning database design and general implementation (MySQL or MS Access): Is it really necessary to have hundreds of columns (two for each sensor) in Table1? I wish I could store the “link” to the respective time series data simply as two additional columns in Table2.
Is that feasible? Does that even make sense? How would I set up this database automatically (coming from .csv files with a different structure) since I can’t do type in every column by hand for hundreds of sensors and their attached time series?
In the end, I want to be able to make a query/sort my data (see below) by timeframe, date and sensor-properties.
The reason for all of this is the following:
I want to create a third table (Table3) which “stores” dynamic values. These values are results of calculations based on the sensor-measurements and WCD in Table 1. However, depending on the sensor-properties in Table2, the sensors and their respective time series data that serve as input for the calculations of Table3 might differ from set to set.
That way I want to obtain e.g. Set 1: “a portfolio of sensors with location A for each month between January 2010 and November 2011” and store it somewhere. Then I want to do the same for Set 2: e.g. “a portfolio of sensors with location B for the same time frame”. Finally I will compare these different portfolios and conduct further analysis on them. Does that sound reasonable at all??
So far, I’m not even sure whether I should actually store the results for each calculation of Table3 in the database or if I output them query and feed them directly into my analyzation tool. What makes more sense?
A more useful structure for your sensor and WCD data would be:
Table SD - Sensor Data
Columns:
Datetime
Sensor
A_value
B_value
With this structure you do not need to store a link to the time series data in Table 2--the Sensor value is the common data that links the tables.
If your weather conditions data all have the same type of values and/or attributes then you should normalize it similarly:
Table WCD - Weather Conditions Data, Normalized
Columns:
Datetime
Weather_condition
Weather_condition_value
From your example, it looks like different weather conditions may have different attributes (or different data types of attributes), in which case the form in which you have the WCD in your Table 1 may be most appropriate.
Storing the results of your calculations in another table sounds like a reasonable thing to do if at least some of your further analysis could be, or will be, done using SQL.
I've a requirement to design a database for an ecommerce app that has vast scope of product categories ranging from pin to plane. All products have different kinds of features. For example, a mobile phone has specific features like memory, camera mega pixel, screen size etc whilst a house has land size, number of storeys and rooms, garage size etc. Such specific features go on and on as much as we've products. Whist all have some common features, there are mostly very different and specific features of all. So, it has gotten bit confusing while designing its database. I'm doing it for the first time.
My query is about database design. Here is what I'm planning to do:
Create a master table with all fields, that tells if a field is common or specific and map them with respective category of the product. All products will have "common" fields but "specific" will be shown only for one category.
table: ALL_COLUMNS
columns:
id,
name,
type(common or specific),
category(phone, car, laptop etc.)
Fetch respective fields from all_columns table while showing the fields on the front.
Store the user data in another table along with mapped fields
table: ALL_USER_DATA
columns:
id,
columnid,
value
I don't know what is the right way and how it is done with established apps and site. So, I'm looking forward if someone could tell if this is the right way of database architecture of an ecommerce app with highly comprehensive and sparse set of categories and features.
Thank you all.
There are many possible answers to this question - see the "related" questions alongside this one.
The design for your ALL_USER_DATA table is commonly known as "entity/attribute/value" (EAV). It's widely considered horrible (search SO for why) - it's theoretically flexible, but imagine finding "airplanes made by Boeing with a wingspan of at least 20 metres suitable for pilots with a new qualification" - your queries become almost unintelligible really fast.
The alternative is to create a schema that can store polymorphic data types - again, look on Stack Overflow for how that might work.
The simple answer is that the relational model is not a good fit for this - you don't want to make a schema change for each new product type your store uses, and you don't want to have hundreds of different tables/columns.
My recommendation is to store the core, common information, and all the relationships in SQL, and to store the extended information as XML or JSON. MySQL is pretty good at querying JSON, and it's a native data type.
Your data model would be something like:
Categories
---------
category_id
parent_category_id
name
Products
--------
product_id
price
valid_for_sale
added_date
extended_properties (JSON/XML)
Category_products
-----------------
category_id
product_id
We are building an analytics engine which has to store attribute preference score for each user. We are expecting 400 attributes and they may change(at what frequency is not known as yet). We are planning to store this in Redshift.
My qs is:
Should we store as 1 row per user with 400 cols(1 column for each attribute)
or should we go for a table structure like
(uid, attribute id, attribute value, preference score) which will be (20-400)rows by 3 columns
Which kind of storage would lead to a better performance in Redshift.
Should be really consider NoSQL for this?
Note:
1. This is a backend for real time application with increasing number of users.
2. For processing, the above table has to be read with entire information of all attibutes for one user i.e indirectly create a 1*400 matrix at runtime.
Please help me which desgin would be ideal for such a use case. Thank you
You can go for tables like given in this example and then use bitwise functions
http://docs.aws.amazon.com/redshift/latest/dg/r_bitwise_examples.html
Bitwise functions are here
For your problem, I would suggest a two table design. Its more pain in the beginning but will help in future.
First table would be a key value kind of first table, which would store all the base data and would be kind of future proof, where you can add/remove more attributes, but this table will continue working.
And a N(400 in your case) column 2nd table. This second table you can build using the first table. For the second table, you can start with a bare minimum set of columns .. lets say only 50 out of those 400. So that querying this table would be really fast. And the structure of this table can be refreshed periodically to match with the current reporting requirements. Also you will always have the base table in case you need to backfill any data.
I am developing an app in cakephp with user auth. Users will add their customer names every time they get orders. So I want to have an auto complete textfield for customer name to add orders. Each user will have their own set of customer names.
So should i create a big text to store customer names(all terms with comma seperated)
/ or /
varchar for each term (1 term in 1 record)?
I will use foreign key to separate users customers.
I am planning to use jQuery ui auto complete with sourcing terms from the customer table values.
My big concern is database capacity, I would like to save the space in database because I have other tables and a lot of users too.
(I do not have a programming background, so please forgive me for my typo)
Thank you.
Use multiple records, one for each term. That is what databases are designed to store.
If you store all the terms in a comma separated list, you will discover that there are lots of things that you cannot easily do.