I'm having some troubles with designing my website. I'm trying to use OOP design in the way I design my site and using MySQL to store the objects in JSON format.
So I'm creating a MySQL table inside my database. The table is going to contain a primary key (PK) and what I call page type (pageType, ex: homePage, aMessage, aTutorial, etc). This means that I will have serval different pages that have different page formats (pageFormats, ex: headerArea, contentArea, footerArea, etc). So depending on the pageType object that was requested, the query would then go to pageFormat table to retrieve the desired divs.
So for example, we have a AJAX request that says the pageType is set to homePage. The request would then go to the pageFormat table and see which divs the homePage is allowed and then return them. I then of course would write them to the document and continue on loading the page with desired content and so forth.
I am just having trouble going from my UML / documentation to actual development of this idea. So if someone could help me with this it'd be greatly appreciated. The trouble that is most difficult for me to understand is that in MySQL database I have is setting up the table for pageType and pageFormat.
The returned types would be in JSON form so the scripts of my page would be able to format them correctly. So that leads me to my second question of what is the best way to store JSON objects in a MySQL table that are going to be divs? Would it be TINYTEXT? Because I don't plan on having large amount of text in there?
Then my last question would be, what would be the best table type? I'm having trouble with selecting this as well.
I have referenced http://dev.mysql.com/doc/refman/5.1/en/storage-engines.html for the table types to try and help me though I'm still unsure.
I also have been reading http://www.agiledata.org/essays/mappingObjects.html#BasicConcepts to understand how to implement relational databases and mapping objects to them. Is there any other good reads that I should look into?
Thanks for any help / direction.
If you want to objects data as JSON-documents, I recommend you to use a NoSQL database like MongoDB or CouchDB instead of MySQL, since they allow you to use JSON directly to store data.
With MySQL, the most common approach is to use a ORM layer to avoid the impedance between a relational database management system and a object-oriented language. By storing a JSON-like object in a RDMS, you are forcing your dynamic schema document to fit in a scalar value, which in my opinion is not the best solution at all.
Related
I am developing an e-commerce website where for each seller there are multiple images for each item. Can any one give me some suggestions on what is the best way/structure to handle the storage of these image links.
Currently I already have some solutions but not sure if they are efficient enough.
1. create another mysql table to associate items and images (I really don't like this solution)
use some special char to delimit image links
store a JSON string containing an array of image links. When I need those links, read the JSON string into an object and do the work.
I'm not good at analyzing I/O. Could any one give me some suggestions and directions?
Why do you don't like solution 1? It's the best and simple solution to your problem.
Option #2 and #3 requires you to read the database field and use some function to read/parse field to get URLs. Inserts or updates to this multivalued fields can be a nightmare and error prone.
Option #1 requires other table with PK/FK relationship, but is the more manageable solution.
Just use your solution #1.
This is a complex problem, so I'm going to try to simplify it.
I have a mysql instance on my server hosting a number of schemas for different purposes. The schemas are structured generally (not perfectly) in a EAV fashion. I need to transition information into and out of that structure on a regular basis.
Example1: in order to present the information on a webpage, I get the information, stick it into a complex object, which I then pass via json to the webpage, where I convert the json into a complex javascript object, which I then present with knockoutjs and similar things.
Conclusion: This resulted in a lot of logic being put into multiple places so that I could associate the values on the page with the values in the database.
Example2: in order to allow users to import information from a pdf, I have a lot of information stored in pdf form fields. In this case, I didn't write the pdf though, so the form fields aren't named in such a way that all of this logic is easy enough to write 3 or more times for CRUD.
Conclusion: This resulted in my copying a list of the pdf form fields to a table in the database, so that I could then somehow associate them with where their data should be placed. The problem that arose is that the fields on the pdf would need to associate with schema.table.column and the only way I found to store that information was via a VARCHAR
Neither of the examples are referring to a small amount of data (something like 6 tables in example 1 and somewhere around 1400 pdf form fields in example 2). Given Example1 and the resulting logic being stored multiple places, it seemed logical to build Example2, where I could store the relationships between data in the database where they could be accessed and changed consistently and for all involved methods.
Now, it's quite possible I'm just being stupid and all of my googling hasn't come across that there's an easy way to associate this data with the correct schema.table.column If this is the case, then telling me the right way to do that is the simple answer here.
However, and this is where I get confused. I have always been told that you never want to store information about a database in the database, especially not as strings (varchar). This seems wrong on so many levels and I just can't figure out if I'm being stupid, and it's better to follow Example1 or if there's some trick here that I've missed about database structure.
Not sure where you got "... never ... store information about a database in the database". With an EAV model it is normal to store the metamodel (the entity types and their allowable attributes) in the database itself so that it is self-describing. If you had to change the metamodel, would you rather change code or a few rows in a table?
The main drawback to EAV databases is that you lose the ability to do simple joins. Join-type operations become much more complex. Like everything else in life, you make tradeoffs depending on your requirements. I have seen self-describing EAV architectures used very successfully.
I have a fairly large database containing a number of different tables representing different product types (eg. cars; baby strollers).
I'm using a website built with PHP to access the data and display it, and I allow users to filter the data (typical online product database sort of stuff).
I'm not sure if I went about storing my metadata the correct way. I'm using XML to do a lot of stuff, which requires making a product type table in MySQL first, and then adding information about each of the columns in that table in my big XML "column attribute" file. So I'll have the name of each column listed in the XML table with information about the column. I store localized names for the column in the XML file, and indicate what type of information about the product is being stored in the column (e.g. Is a column showing a dimension (to be listed in the product dimensions area) or a feature (for the features area)).
First off, am I way off base storing all this custom metadata in XML?
Secondly, if I should be storing some of it in MySQL (and I think I should be moving some of it there), what's the best way to do that? I see that I can make column "comments" in MySQL....are those standard fare for databases? If I move to Oracle some day, would I lose all my comment info? I'm not thinking of moving much information to the database, and some of it could be accomplished by just adding a little identifier to my column names (e.g. number_of_wheels becomes number_of_wheels_quantity, length becomes length_dimension)
Any advice from the database design gurus out there would be vastly appreciated. Thanks :)
First off, am I way off base storing all this custom metadata in XML?
Yes, XML is a great markup for transporting data in a nearly human readable format, but a horrible one for storing it. It's very costly to search through XML, and I don't know of a (good) way to have a query search through XML stored in a field in the DB. You are probably better off with a table that stores these things directly, you can easily convert them into XML if you need to, after you query them from the DB. I think in your case a table with the following columns would be useful: "ColumnName","MetaData" Would be all you need, populate with values as per your example:
__________________________________________________________________________________________________
|colDimension | Is a column showing a dimension (to be listed in the product dimensions area) |
|colFeature | a feature (for the features area) |
--------------------------------------------------------------------------------------------------
This scheme will resolve your comments conundrum as well, as you can add another field to the above table to store the comments in, which will make them much more accessible to your middle tier (php in your case) if you ever want to display those comments.
I had to make a few assumptions as to intent and existing data and whatnot, so if I'm wrong about anything, let me know why it doesn't work for you and I'll respond with some corrections or other pointers.
See, your purpose is to keep the Meta data at a place. right?
I'll suggest you to use the freely available tool Mysql Workbench. In this tool you have option to create ER diagram (or EER diagram). You can keep the whole structure and at any point of time you can sync with server and restore the structure. You can backup those structures also. Its kind of you have to learn first if you are not already using it. But at last its a very helpful tool for keeping the structure in an organized way.
I'm building a PHP/MySQL website and I'm currently working on my database design. I do have some database and MySQL experience, but I've never structured a database from scratch for a real world application which hopefully is going to get some good traffic, so I'd love to hear advices from people who've already done it, in order to avoid common mistakes. I hope my explanations are not too confusing.
What I need
In my application, the user should be able to write a post (title + text), then create an "object" (which can be anything, like a video, or a song, etc.) and attach it to the post. The site has a list of predefined object types the user can create, and I should be able to add new types in the future. The user should also have the ability to see the object's details in a dedicated page and add a comment to it - the same applies to posts.
What I tried
I created an objects table with these fields: oid, type, name and date. This table contains records for anything the user should be able to add comments to (i.e. posts and objects). Then I created a postmeta table which contains additional post data (such as text, author, last edit date, etc.), a videometa table for data about the "video" object (URL, description, etc.), and so on. A postobject table (pid,oid) links objects to posts. Additionally, there's a comments table which contains the comment text, the author and the ID of the object it refers to.
Since the list of object types is predefined and is probably not going to change (though I still need the ability to add a type easily at any time without changing the app's code structure or the database design), and it is relatively small, it's not a problem to create a "meta" table for each type and make a corresponding PHP class in my application to handle it.
Finally, a page on the site needs to show a list of all the posts including the objects attached to it, sorted by date. So I get all the records from the objects table with type "post" and join it with postmeta to get the post metadata. Then I query postobject to get all the objects attached to this post, and comments to get all the comments.
The questions
Does this make any sense? Is it any good to design a database in this way for a real world site? I need to join quite a few tables to get all the data I need, and the objects table is going to become huge since it contains almost every item (only the type, name and creation date, though) - this is to keep the database and the app code flexible, but does it work in the real world, or is it too expensive in the long term? Am I thinking about it in the wrong way with this kind of OOP approach?
More specifically: suppose I need to list all the posts, including their attached objects and metadata. I would need to join these tables, at least: posts, postmeta, postobject and {$objecttype}meta (not to mention an users table to get all posts by a specific user, for example). Would I get poor performance doing this, even if I'm using only numeric indexes?
Also, I considered using a NoSQL database (MongoDB) for this project (thanks to Stuart Ellis advice). Apparently it seems much more suitable since I need some flexibility here. But my doubt is: metadata for my objects includes a lot of references to other records in the database. So how would I avoid data duplication if I can't use JOIN? Should I use DBRef and the techniques described here? How do they compare to MySQL JOINs used in the structure described above in terms of performance?
I hope these questions do make any sense. This is my first project of this kind and I just want to avoid to make huge mistakes before I launch it and find out I need to rework the design completely.
I'm not a NoSQL person, but I wonder whether this particular case might actually be handled best with a document database (MongoDB or CouchDB). Various type of objects with metadata attached sounds like the kind of scenario that MongoDB is designed for.
FWIW, you've got a couple of issues with your table and field naming that might bite you later. For example, type and date are rather generic, and also reserved words. You've also mixed singular and plural table names, which will throw any automatic object mapping.
Whichever database you use, it's a good idea to find an existing set of database naming conventions and apply it from the start - this will help you avoid subtle issues and ensure that your naming stays consistent. I tend to use the Rails naming conventions ATM, because they are well-known and fairly sensible.
Or you could store the object contents as a file, outside of the database, if you're concerned about the database space.
If you store anything in the database, you already have the object type in objects; so you could just add object_contents table with a long binary field to store the object. You don't need to create a new table for each new type.
I've seen a lot of JOIN's in real world web application (5 to 10). Objects table may get large, but that's indices are for. So far, I don't see anything wrong in your database. BTW, what felt strange to me - one post, one object, and separate comments for each? No ability to mix pictures with text?
I'm developing a form generator, and wondering if it would be bad mojo to store JSON in an SQL database?
I want to keep my database & tables simple, so I was going to have
`pKey, formTitle, formJSON`
on a table, and then store
{["firstName":{"required":"true","type":"text"},"lastName":{"required":"true","type":"text"}}
in formJSON.
Any input is appreciated.
I use JSON extensively in my CMS (which hosts about 110 sites) and I find the speed of access data to be very fast. I was surprised that there wasn't more speed degradation. Every object in the CMS (Page, Layout, List, Topic, etc) has an NVARCHAR(MAX) column called JSONConfiguration. My ORM tool knows to look for that column and reconstitute it as an object if needed. Or, depending on the situation, I will just pass it to the client for jQuery or Ext JS to process.
As for readability / maintainability of my code, you might say it's improved because I now have classes that represent a lot of the JSON objects stored in the DB.
I used JSON.net for all serialization / deserialization. https://www.newtonsoft.com/json
I also use a single query to return meta-JSON with the actual data. As in the case of Ext JS, I have queries that return both the structure of the Ext JS object as well as the data the object will need. This cuts out one post back / SQL round trip.
I was also surprised at how fast the code was to parse a list of JSON objects and map them into a DataTable object that I then handed to a GridView.
The only downside I've seen to using JSON is indexing. If you have a property of the JSON you need to search, then you have to store it as a separate column.
There are JSON DB's out there that might server your needs better: CouchDB, MongoDB, and Cassandra.
A brilliant way to make an object database from sql server. I do this for all config objects and everything else that doesn't need any specific querying. extending your object - easy, just create a new property in your class and init with default value. Don't need a property any more? Just delete it in the class. Easy roll out, easy upgrade. Not suitable for all objects, but if you extract any prop you need to index on - keep using it. Very modern way of using sql server.
It will be slower than having the form defined in code, but one extra query shouldn't cause you much harm. (Just don't let 1 extra query become 10 extra queries!)
Edit: If you are selecting the row by formTitle instead of pKey (I would, because then your code will be more readable), put an index on formTitle
We have used a modified version of XML for exactly the purpose you decribe for seven or eight years and it works great. Our customers' form needs are so diverse that we could never keep up with a table/column approach. We are too far down the XML road to change very easily but I think JSON would work as well and maybe evan better.
Reporting is no problem with a couple of good parsing functions and I would defy anyone to find a significant difference in performance between our reporting/analytics and a table/column solution to this need.
I wouldn't recommend it.
If you ever want to do any reporting or query based on these values in the future it's going to make your life a lot harder than having a few extra tables/columns.
Why are you avoiding making new tables? I say if your application requires them go ahead and add them in... Also if someone has to go through your code/db later it's probably going to be harder for them to figure out what you had going on (depending on what kind of documentation you have).
You should be able to use SisoDb for this. http://sisodb.com
I think it not an optimal idea to store object data in a string in SQL. You have to do transformation outside of SQL in order to parse it. That presents a performance issue and you lose the leverage of using SQL native data parsing capability. A better way would be to store JSON as an XML datatype in SQL. This way, you kill two birds with one stone: You don't have to create shit load of tables and still get all the native querying benefits of SQL.
XML in SQL Server 2005? Better than JSON in Varchar?