I am learning Meteor using the Discover Meteor book.
I come from a PHP and MySQL background, and the application I am thinking of doing as a side-project is a real-time Backgammon web application. While Meteor's reactivity is a very, very big plus, I am stumped on how I can handle relational data (e.g. games, users, tournaments, friends, teams, etc).
I have read a lot of answers (ranging from old to very old) on StackOverflow on how one can use MySQL with Meteor. My search has led me to numtel/meteor-mysql. However, when I look at the examples provided in that repository, it is nowhere as clean as Meteor's own implementation of MongoDB.
My options, as I understand them, are the following:
Use MongoDB, and rewrite a lot of the features present in RDBMS in Javascript
Use an RDBMS that is not as well-supported in Meteor as MongoDB
IMO, option two is much less work, and I think might lead to less problems in the future. Take the problem in the epilogue of Why You Should Never Use MongoDB, for example.
We could also model this data as a set of nested hashes. The set of information about a particular TV show is one big nested key/value data structure. Inside a TV show, there’s an array of seasons, each of which is also a hash. Within each season, an array of episodes, each of which is a hash, and so on. This is how MongoDB models the data. Each TV show is a document that contains all the information we need for one show.
But then, how would you query for the TV shows that someone has starred in?
Back to my original question: is there something I'm missing here? Handling relational data is something that a lot of applications will need to do, but I can't seem to find a clean solution
It will be much less work if you go with option 1 in my opinion.
It won't be difficult to learn to use MongoDB, and since MongoDB uses JSON objects and is supported natively by Meteor and all it's packages, it will be much less work.
I advise having a look at the aldeed packages: collection2 and simple-schema to structure your collections. I also advice using the collection-helpers package to help with joins.
If you have a posts collection with name, authorId and content fields, then to get the author of the post, you'd write Meteor.users.findOne(userId).
Hope that clears things up a bit and gets you on your way.
Related
We are working with a DDD framework in our company. We are changing a lot of core things in our API because we are still growing and we are still in our enfant phase when designing a good API.
The problem is that there are alot of flows already in the same api. Which are not compatible with eachother.
We have an order service and a product service.
Normally when the product model radically changes, we have a major impact in the order model.
Now im here listing all kind of red flags which should never happen but I simply dont have control over how it needs to be done. That is pretty much management pushing for a fast solution. And leading to bad shortcuts...
The way is has been decided to overcome that Order needs to adapt constantly. They made a property in the orderline called productConfiguration. This is in the contract of the service and is direcrtly translated as is in the DB tables. This contains the product model that can change. In json format.
For me its very clear that this is very dangerous to do this. Because i nthe end you need to change this json into an actual object. So you just move the restrictions from the service contract to code logic. Which makes it worse cause it will only cause an issue at run time...
Are there other major things I just know about, so I can bring it to the table to avoid this way of working...
Using strings that are directly converted into DB tables is not just in your opinion a bad design. It's an opinion shared by a lot of us.
What do you do when an object changes? For example, the new one requires an attribute that the old one didn't had. How do you manage this situation? I suppose that you've to change everything, including the objects stored before. Or build a kind of transformation layer where you translate objects from the old to the new design. A lot of extra work.
Anyway, given that the two domains are separated, what are the information that change so much and require such a design? I mean, for most of the things you could know at the beginning what do you need for your part of the domain. For the rest, I would prefer to have a kind of service that given an Id gives you the information from the other domain. You can change this service (here could be also json obj, if nothing than just showing is required) and adapt to your/their needs. But, it's just a solution that comes from my limited knowledge of your processes.
Other ways are also possible, as long as you can always understand which version of the design are you using.
I am creating a webpage in ReactJS for post feed (with texts, images, videos) just like Reddit with infinite scrolling. I have created a single post component which will be provided with the required data. I am fetching the multiple posts from MySQL with axios. Also, I have implemented redux store in my project.
I have also added post voting. Currently, I am storing all the posts from db in redux store. If user upvotes or downvotes, that change will be in redux store as well as in database, and web-page is re-rendering the element at ease.
Is it feasible to use redux-store for this, as the data will be increased soon, maybe in millions and more ?
I previously used useState hook to store all the data. But with that I had issue of dynamic re-rendering, as I had to set state every time user votes.
If anyone has any efficient way, please help out.
Seems that this question goes far beyond just one topic. Let's break it down to the main pieces:
Client state. You say that you are currently using redux to store posts and update the number of upvotes as it changes. The thing is that this state is not actually a state in your case(or at least most of it). This is a common misconception to treat whatever data that is coming from API a state. In most cases it's not a state, it's a cache. And you need a tool that makes work with cache easier. I would suggest trying something like react-query or swr. This way you will avoid a lot of boilerplate code and hand off server data cache management to a library.
Infinite scrolling. There are a few things to consider here. First, you need to figure out how you are going to detect when to preload more posts. You can do it by using the IntersectionObserver. Or you can use some fance library from NPM that does it for you. Second, if you aim for millions of records, you need to think about virtualization. In a nutshell, it removes elements that are outside of the viewport from the DOM so browsers don't eat up all memory and die after some time of doomscrolling(that would be a nice feature tho). This article would be a good starting point: https://levelup.gitconnected.com/how-to-render-your-lists-faster-with-react-virtualization-5e327588c910.
Data source. You say that you are storing all posts in database but don't mention any API layer. If you are shooting for millions and this is not a project for just practicing your skills, I would suggest having an API between the client app and database. Here are some good questions where you can find out why it is not the best idea to connect to database directly from client: one, two.
Does anybody have any experience with as3-spod?
I downloaded the source code from github and as3-signals and started to try it out, but I´ll take ages to get to know the framework by trial and error and probably miss a lot of best practices. The framework looks good but lack's on examples. The git page does't have a lot of info on that...
If anybody knows some other ORM for AIR that I can use on pure AS3 projects that have any bit of documentation, I´m more than thankful!
I was hoping to do a question-comment asking for clarification, but I don't have enough reputation yet! So I will answer as best I may.
I am using as3-spod for my application. It's been pretty reliable and mostly given me what I want. It's not really ideal, though. What I'd really like is something more ActiveRecord-like, or something original that lets you generate queries by concatenating conditions in a fluid syntax.
But if you're not using Flex (as I'm not, and you're not) then your options are pretty thin, as most of the other AS3 ORMs out there rely on some part of the Flex framework. Apart from as3-spod, the only possibility I could find was Christophe Coenraets' proof-of-concept but as he points out, it would need a lot of work to develop it into a fully-fledged ORM:
This is still a simplistic proof of concept and is by no means a production ready ORM solution.
And I haven't had time for that.
You are right that as3-spod is quite poorly documented. I guess the main class you want to look at is SpodTable. It's from that one you do inserts, selects, etc. An update on a single object can be done from the object itself. Look out for the various signals on SpodTable (select, selectAll, etc). To get going with it, just mark up a model class with metadata, then from your SpodDatabase instance call createTable(MyModelClass).
My main gripes with as3-spod are these (I'm listing them so you don't look for features that don't exist, which I wasted a fair bit of time doing!):
It works asynchronously. Doesn't matter if your actual SQLConnection has been opened synchronously or asynchronously; you have to listen to signals. That means you can't retrieve records and then use them straight away in the same method, you have to listen to signals. What I tend to do is to do large selects when the app starts, then filter the data in memory rather than doing complex queries. Pretty annoying.
Be careful with null values for numeric columns. I can't see a way of setting NULL or NOT NULL for columns using as3-spod; it always seems to make them NOT NULL, which will cause errors if you try to insert a row from an object with null fields.
There's no migration system (a la Rails). I am working on rolling my own as that's an essential feature for my purposes (it's a mobile app I'm developing).
Good luck! Let me know in comments if there's anything else specific you'd like me to cover and I can expand this answer.
EDIT
I've just noticed the existence of AS3SQLite. Haven't used it yet but, looks like there are other possibilities out there :)
I know this is a somewhat subjective question but I want to throw it out there anyway because if there is some insight that I've missed, asking this question will save me (and hopefully others) a great deal of searching :-)
Basically, what is the best type of database for when you have a number of items interlinked. For example:
A->B->C->D->E->F->G->H->I
And you want to be able to quickly find the shortest path (or the number of paths ideally) between A and I?
I would normally just use a relational database for this, but I'm not sure a MapReduce style database is any better and then I wondered if there was a type of database I hadn't considered altogether...
As always, all help gratefully received :)
This is exactly the sort of thing neo4j is designed for. It's a graph database that provides operations for doing graph traversal and other common operations.
I would suggest looking into MongoDB - It's NOT a relational database, and all your data gets saved and passed as JSON, it's easy scale-able and build for performance, and comes with a couple of tricks up its sleeve as well, like MapReduce build into mongoDB and way way more.
Since the data is saved as JSON you can save objects within objects and arrays well:
{
foo:"bar",
animals: ['cat','dog','fish'],
blog:{
post:"Hello World!",
comments:["this is cool","Hello back"]
}
}
Checkout - http://www.mongodb.org/
or http://www.mongodb.org/display/DOCS/MapReduce/
for more info
Am I completely off-track to think about using Semantic MediaWiki to store (and organise, report on, etc.) 'tabular' data such as financial transactions or weather readings that would usually live in a spreadsheet or database?
It seems that one would need a separate, tiny, page for each tuple; but then, that's by design and perhaps it's perfectly okay.
I ask, simply because SMW seems like such a quick and easy way to get a collaborative data repository up and running.
Semantic MediaWiki is better suited for keeping track of Factual or Encyclopedic data, where you can have pages about everything you need to know about a certain topic.
For tabular or numerical data such as measurements, financial, sensor data, you would indeed need to create little pages about each data point, which is not practical in many cases.
However, there are extensions to Media Wiki that allow you to integrate external data sources (in MySQL databases or CSV files somewhere) with MediaWiki pages. This can allow you to have the best of both worlds - dynamic access and queries of tabular data and semantic annotations of pages around them.
Take a look at :
http://www.mediawiki.org/wiki/Extension:External_Data
No, I don't think it's such a bad idea.
Using SemanticForms you could enter lots of little data pages quickly and easily (for example, an invoice might require additional pages for each line item, but they could all be entered from one form using the 'multiple' feature of the for template form tag). So although I've never tried logging weather data in SMW, I think it would be pretty easy. I don't see what the problem would be with storing data across so many pages; it's easy enough to combine it in whatever formats you require.
Give it a go and let us know how it goes!
You can use either the Semantic Internal Objects extension (SIO), or SMW's built in subobjects (the former works well with the already mentioned External Data extension), to store multiple semantic objects (could be the rows of your spreadsheet) in one page.
However, unless you are really looking for a collaborative tool with semantic capabilities, I doubt SMW is the best suited piece of software for your task.
edit (november 2015): Since SMW version 1.9, there nothing that SIO can do that the built-in subobjects can't, so I would recommend the latter.