I am working on an embedment section on my site where users can embed different media from various services, youtube, myspace music, vimeo etc
I am trying to work out the best way to store it. Users do not have to embed all of the options and can only embed one of each type (one video for example).
Initially I thought just have a table with a row per embeded item like so:
embedid (auto increment primary key),
userid, embedded_item_id (e.g a
youtube id)
but then I realised that some embeddable items require multiple arguments such as myspace music so I thought id make a table where each user has one row.
userid, youtubeid, vimeoid,
myspaceid1, myspaceid2
but it seems a bit clumsy especially considering there will always be empty rows as users can not ever have all of them. Does anyone have a better solution?
`EmbededItem' table has columns common to all items.
YouTube, Vimeo, MySpace have only columns specific to each one.
So, here's what I'd do in such a situation:
Setup your table with columns for your primary key and userid fields and anything else you may need to identify the user or application (maybe a 'mediatype' field). The rest, put into a VARCHAR field, make it large enough to hold lots of data. Not sure how much space you would need, but I'm going to venture a guess that you will need between 1K and 4K+ of space.
The reason for a VARCHAR field: you never know what other new fields you will need in the future. Let's say next year youtube adds another parameter, or a new media format comes along. If you model your database to represent all fields individually, you will create an application that is not scalable to future or other media formats. Such modeling is great when you're describing a system on paper, but not so good when you implement code.
So, now that you have a varchar field to store all your data in, you have several options for how to store the data:
You can store the data as an XML document and parse it on input/output (But you will most likely need more than 4k of space), and you will incur the cost of parsing XML.
You can store the data as whatever data format you may need for your application (serialized object for java, JSON for javascript, etc). If you're serializing an object, you may also need more than 4k of space, and a VARBINARY field, not VARCHAR.
comma delimited string, although this fails if your strings contain commas. I probably would not recommend this.
null delimited key/value pair strings, with a double null at the end. You will need a VARBINARY data field for this one.
Number 4 is my favorite, and something I would recommend. I've used this pattern for an existing web project, where my strings are stored in format of:
'uid=userid/0var1=value1/0val2=value2/0url=urltosite/0/0'
Works like a charm. I use the data to build dynamic web pages for my users. (My application is C though, so it deals well with parsing a character array).
Your application could use the data from your first columns (like 'mediatype') to execute specific parsing routines if required, and use the VARCHAR/VARBINARY fields as input. Scaling to new types of embeddable media would be as simple as writing a new (or extending an existing) parser and defining a new 'mediatype' value.
Hope this helps.
Related
I need to store for each row of a table possible references to 5 other tables because I need to know, for each one of five tables, if are there references to the row .
I was thinking about using a "binary type" code so for each table I'll have a "0/1" and for five tables I'll have from "00000" (for no references) to "11111" (for five references).
So, if I only have references into table 3 I'll store "00100".
Now here are my questions:
1) Is it a good idea? Are there better solutions?
2) What kind of field do I need to use? (I was thinking ENUM)
EDIT (TO CLARIFY)
I need to know if I'll need to access (or not) to each of five table to get data related to the row.
The five tables are web tables so I need to know if I can find informations on a table or not.
EDIT 2 (further clarification)
I don't query the web tables: I get info from them by code. The code reads from my DB to know how many web tables needs to access and which of the 5 existing.
I would likely store the information in a Relational Database Management System relationally; as close to 3rd normal form or better as possible. This means I would have two tables.
One table lists the available options. A second table lists all the options affiliated with that object which is accessing the 5 web tables.
The reason I would do it this way has to do with being able to expand the application. If additional web tables become available, or if other objects over time need access to different web objects, this model would support growth both in number of web objects, and objects accessing the web objects without having to touch code.
Now, if that's GUARANTEED not to happen, then I could see storing the values in a single field where position matters; perhaps using a "SET" data type. But given there are few guarantees I would likely not take this approach just because it's not scalable without code change and doesn't perform as well under volume.
To address your specific questions:
1) Is it a good idea? Are there better solutions?
Personally I find the idea poor in that it can not easily scale without code change; and I don't see how an ENUM can be stored in one field without using set data types. (The two work well together but I don't think I've seen ENUM without set when storing multiple values in one field) I prefer a more relational method. However, if we know that it will never grow, then I find the approach reasonable.
2) What kind of field do I need to use? (I was thinking ENUM)
I would prefer SET Datatype. I think it would be easier to scale and is more Consistent. I'm not sure how you would multi select the ENUM data type; without storing it relationally or using set data types. If you're adding a new table to store the ENUM values then that's more or less the same as what I would recommend.
But my exposure and use to ENUM is limited in this context; so I may not have some experience which would make this a reasonable approach.
I have a table in My Sql qhere i have fields lie name, location, description and picture.
What I want to do is store multiple picture links in the picture column.
Is there a way of doing that without creating a separate table for picture?
Thank you
Well you need to perform some sort of serialization in order to do that. I used to do that before I moved to document-oriented databases. Quite possibly your best option is to store everything in a json format as it is pretty universal and I can't think of any language that cannot handle it and convert it back to an object, array, dictionary or whatever the language requires. Assuming you need to save the name of the file as in somefile.png, what you could do is store ["image1.png","image2.png","image3.png"] and so on. If you want to store a blob however it's a bit more complicated. You either have to create a second table or read the contents of each image, convert it to base64, load all base64 strings into an object and then serialize it into a json. I wouldn't recommend that as each operation would cost a lot of system resources.
Let's assume I need to store some data of unknown amount within a database table. I don't want to create extra tables, because this will take more time to get the data. The amount of data can be different.
My initial thought was to store data in a key1=value1;key2=value2;key3=value3 format, but the problem here is that some value can contain ; in its body. What is the best separator in this case? What other methods can I use to be able to store various data in a single row?
The example content of the row is like data=2012-05-14 20:07:45;text=This is a comment, but what if I contain a semicolon?;last_id=123456 from which I can then get through PHP an array with corresponding keys and values after correctly exploding row text with a seperator.
First of all: You never ever store more than one information in only one field, if you need to access them separately or search by one of them. This has been discussed here quite a few times.
Assuming you allwas want to access the complete collection of information at once, I recommend to use the native serialization format of your development environment: e.g. if it is PHP, use serialze().
If it is cross-plattform, JSON might be a way to go: Good JSON encoding/decoding libraries exist for something like all environments out there. The same is true for XML, but int his context the textual overhead of XML is going to bite a bit.
On a sidenote: Are you sure, that storing the data in additional tables is slower? You might want to benchmark that before finally deciding.
Edit:
After reading, that you use PHP: If you don't want to put it in a table, stick with serialize() / unserialize() and a MEDIUMTEXT field, this works perfectly, I do it all the time.
EAV (cringe) is probably the best way to store arbitrary values like you want, but it sounds like you're firmly against additional tables for whatever reason. In light of that, you could just save the result of json_encode in the table. When you read it back, just json_decode to get it back into an array.
Keep in mind that if you ever need to search for anything in this field, you're going to have to use a SQL LIKE. If you never need to search this field or join it to anything, I suppose it's OK, but if you do, you've totally thrown performance out the window.
it can be the quotes who separate them .
key1='value1';key2='value2';key3='value3'
if not like that , give your sql example and we can see how to do it.
I have a fairly large database containing a number of different tables representing different product types (eg. cars; baby strollers).
I'm using a website built with PHP to access the data and display it, and I allow users to filter the data (typical online product database sort of stuff).
I'm not sure if I went about storing my metadata the correct way. I'm using XML to do a lot of stuff, which requires making a product type table in MySQL first, and then adding information about each of the columns in that table in my big XML "column attribute" file. So I'll have the name of each column listed in the XML table with information about the column. I store localized names for the column in the XML file, and indicate what type of information about the product is being stored in the column (e.g. Is a column showing a dimension (to be listed in the product dimensions area) or a feature (for the features area)).
First off, am I way off base storing all this custom metadata in XML?
Secondly, if I should be storing some of it in MySQL (and I think I should be moving some of it there), what's the best way to do that? I see that I can make column "comments" in MySQL....are those standard fare for databases? If I move to Oracle some day, would I lose all my comment info? I'm not thinking of moving much information to the database, and some of it could be accomplished by just adding a little identifier to my column names (e.g. number_of_wheels becomes number_of_wheels_quantity, length becomes length_dimension)
Any advice from the database design gurus out there would be vastly appreciated. Thanks :)
First off, am I way off base storing all this custom metadata in XML?
Yes, XML is a great markup for transporting data in a nearly human readable format, but a horrible one for storing it. It's very costly to search through XML, and I don't know of a (good) way to have a query search through XML stored in a field in the DB. You are probably better off with a table that stores these things directly, you can easily convert them into XML if you need to, after you query them from the DB. I think in your case a table with the following columns would be useful: "ColumnName","MetaData" Would be all you need, populate with values as per your example:
__________________________________________________________________________________________________
|colDimension | Is a column showing a dimension (to be listed in the product dimensions area) |
|colFeature | a feature (for the features area) |
--------------------------------------------------------------------------------------------------
This scheme will resolve your comments conundrum as well, as you can add another field to the above table to store the comments in, which will make them much more accessible to your middle tier (php in your case) if you ever want to display those comments.
I had to make a few assumptions as to intent and existing data and whatnot, so if I'm wrong about anything, let me know why it doesn't work for you and I'll respond with some corrections or other pointers.
See, your purpose is to keep the Meta data at a place. right?
I'll suggest you to use the freely available tool Mysql Workbench. In this tool you have option to create ER diagram (or EER diagram). You can keep the whole structure and at any point of time you can sync with server and restore the structure. You can backup those structures also. Its kind of you have to learn first if you are not already using it. But at last its a very helpful tool for keeping the structure in an organized way.
I'm building a PHP/MySQL website and I'm currently working on my database design. I do have some database and MySQL experience, but I've never structured a database from scratch for a real world application which hopefully is going to get some good traffic, so I'd love to hear advices from people who've already done it, in order to avoid common mistakes. I hope my explanations are not too confusing.
What I need
In my application, the user should be able to write a post (title + text), then create an "object" (which can be anything, like a video, or a song, etc.) and attach it to the post. The site has a list of predefined object types the user can create, and I should be able to add new types in the future. The user should also have the ability to see the object's details in a dedicated page and add a comment to it - the same applies to posts.
What I tried
I created an objects table with these fields: oid, type, name and date. This table contains records for anything the user should be able to add comments to (i.e. posts and objects). Then I created a postmeta table which contains additional post data (such as text, author, last edit date, etc.), a videometa table for data about the "video" object (URL, description, etc.), and so on. A postobject table (pid,oid) links objects to posts. Additionally, there's a comments table which contains the comment text, the author and the ID of the object it refers to.
Since the list of object types is predefined and is probably not going to change (though I still need the ability to add a type easily at any time without changing the app's code structure or the database design), and it is relatively small, it's not a problem to create a "meta" table for each type and make a corresponding PHP class in my application to handle it.
Finally, a page on the site needs to show a list of all the posts including the objects attached to it, sorted by date. So I get all the records from the objects table with type "post" and join it with postmeta to get the post metadata. Then I query postobject to get all the objects attached to this post, and comments to get all the comments.
The questions
Does this make any sense? Is it any good to design a database in this way for a real world site? I need to join quite a few tables to get all the data I need, and the objects table is going to become huge since it contains almost every item (only the type, name and creation date, though) - this is to keep the database and the app code flexible, but does it work in the real world, or is it too expensive in the long term? Am I thinking about it in the wrong way with this kind of OOP approach?
More specifically: suppose I need to list all the posts, including their attached objects and metadata. I would need to join these tables, at least: posts, postmeta, postobject and {$objecttype}meta (not to mention an users table to get all posts by a specific user, for example). Would I get poor performance doing this, even if I'm using only numeric indexes?
Also, I considered using a NoSQL database (MongoDB) for this project (thanks to Stuart Ellis advice). Apparently it seems much more suitable since I need some flexibility here. But my doubt is: metadata for my objects includes a lot of references to other records in the database. So how would I avoid data duplication if I can't use JOIN? Should I use DBRef and the techniques described here? How do they compare to MySQL JOINs used in the structure described above in terms of performance?
I hope these questions do make any sense. This is my first project of this kind and I just want to avoid to make huge mistakes before I launch it and find out I need to rework the design completely.
I'm not a NoSQL person, but I wonder whether this particular case might actually be handled best with a document database (MongoDB or CouchDB). Various type of objects with metadata attached sounds like the kind of scenario that MongoDB is designed for.
FWIW, you've got a couple of issues with your table and field naming that might bite you later. For example, type and date are rather generic, and also reserved words. You've also mixed singular and plural table names, which will throw any automatic object mapping.
Whichever database you use, it's a good idea to find an existing set of database naming conventions and apply it from the start - this will help you avoid subtle issues and ensure that your naming stays consistent. I tend to use the Rails naming conventions ATM, because they are well-known and fairly sensible.
Or you could store the object contents as a file, outside of the database, if you're concerned about the database space.
If you store anything in the database, you already have the object type in objects; so you could just add object_contents table with a long binary field to store the object. You don't need to create a new table for each new type.
I've seen a lot of JOIN's in real world web application (5 to 10). Objects table may get large, but that's indices are for. So far, I don't see anything wrong in your database. BTW, what felt strange to me - one post, one object, and separate comments for each? No ability to mix pictures with text?