I am currently playing around with feathers to see if its a good replacement for meteor.js in our POS back-office application.
It looks really great so far but i cam up with a thing i could not solve.
Sometimes i miss a simple thing if i do not find something on the net but I hope someone can help me to figure out:
The problem:
I am using feathers client and a observable client side to populate a data-grid with documents from the mongoose/mongodb based service. It works fine beside I make an UPDATE/PATCH on ANY of the documents contained in the "table".
That moment all (filtered) find results are replaced by just this updated document.
I know that i could cache the results in an array and then use the respective events to update the array, but i wonder if there is an easier way (as in meteor) that the find result remains intact and if it contains the updated document that is simply updated.
Any help appreciated
Greets from Manila
Ralph
How to keep the data up to date depends much on the frontend framework you are using.
The most framework agnostic solution is feathers-reactive which uses RxJS and works well with React (see this video) and Angular 2+
feathers-vuex for VueJS
can-connect-feathers for CanJS
For everything else you can keep the data up to date manually through the real-time events as you already mentioned.
Related
I am creating a webpage in ReactJS for post feed (with texts, images, videos) just like Reddit with infinite scrolling. I have created a single post component which will be provided with the required data. I am fetching the multiple posts from MySQL with axios. Also, I have implemented redux store in my project.
I have also added post voting. Currently, I am storing all the posts from db in redux store. If user upvotes or downvotes, that change will be in redux store as well as in database, and web-page is re-rendering the element at ease.
Is it feasible to use redux-store for this, as the data will be increased soon, maybe in millions and more ?
I previously used useState hook to store all the data. But with that I had issue of dynamic re-rendering, as I had to set state every time user votes.
If anyone has any efficient way, please help out.
Seems that this question goes far beyond just one topic. Let's break it down to the main pieces:
Client state. You say that you are currently using redux to store posts and update the number of upvotes as it changes. The thing is that this state is not actually a state in your case(or at least most of it). This is a common misconception to treat whatever data that is coming from API a state. In most cases it's not a state, it's a cache. And you need a tool that makes work with cache easier. I would suggest trying something like react-query or swr. This way you will avoid a lot of boilerplate code and hand off server data cache management to a library.
Infinite scrolling. There are a few things to consider here. First, you need to figure out how you are going to detect when to preload more posts. You can do it by using the IntersectionObserver. Or you can use some fance library from NPM that does it for you. Second, if you aim for millions of records, you need to think about virtualization. In a nutshell, it removes elements that are outside of the viewport from the DOM so browsers don't eat up all memory and die after some time of doomscrolling(that would be a nice feature tho). This article would be a good starting point: https://levelup.gitconnected.com/how-to-render-your-lists-faster-with-react-virtualization-5e327588c910.
Data source. You say that you are storing all posts in database but don't mention any API layer. If you are shooting for millions and this is not a project for just practicing your skills, I would suggest having an API between the client app and database. Here are some good questions where you can find out why it is not the best idea to connect to database directly from client: one, two.
I am a beginner web developer and working on a school project. My apologies for asking this basic question.
I am trying to create an online shopping store. What I am trying to do is when the user clicks on the checkout button, the shopping cart page gets updated with that element.
I don't know how to accomplish that. I really appreciate if someone can point me to a tutorial or provides some tips.
Your question is really abstract, however you will have to find the right way to communicate with a database...it might be php/MySQL with laravel or some other framework or preferably C# asp.net with entity framework(that's what I will go with its not mandatory some people like php) Then you need to learn some lambda or sql depending on which approach you would like to take (php/C#). You will probably need to get some kind of grid control for the cart asp.net has default one personally I find it confusing and hard to work with( there are many controls that can be downloaded freely without charge for non commercial products search google) then you would need to fill that grid with the List of objects that you will get from your database. If you get that far alone the next steps would be easy you will probably need some javascript for ajax calls to a webservice or either your backend so you can submit the orders. For the right tutorial there is none actually since your question covers several topics regarding database modeling, data fetching, backend development and front end development. You can start by getting yourself some kind of server (look for mssql for asp.net) and MySQL for php, then you should look for some kind of database modeling tutorial basic tables relationships 1-1 1-* *-1, after that you would need to do your design (from the question I assume that you already have one downloaded). Then if you got down to this step you should google the way for the right way of communicating with your database from the backend of your website (I advise entity framework its clean and easy) you can generally bind the list of objects directly do the grid that would represent your cart (I say grid but you can give it data template to look like html, not to be confused with regular tables). If you really get here I see no reason to keep coding for a school project in most places you can use all this to bachelor degree exam and pass with straight A. But if you want more details look into these videos they explain quite well what you want to do.
https://www.youtube.com/watch?v=s91pPLx_T3Q
https://www.youtube.com/watch?v=yYr0seXj7qA
https://www.youtube.com/watch?v=lFkXk5gHjSs
I am not great at JavaScript but I think your page would need to update a cookie then read that value from said cookie. I hope that's a good place to start!
You might also find "Creating a Shopping Cart using only HTML/JavaScript" helpful.
6.30.15 - HOW CAN I MAKE THIS QUESTION BETTER AND MORE HELPFUL TO OTHERS? FEEDBACK WOULD BE HELPFUL. THANKS!
I am developing a web application that will handle/manage a VERY LARGE data set - Currently any kind of heavy load causes the browser to lock up - whether I'm in the Django Rest Framework API or in the Dojo/Dgrid. This is kind of a dual question.
I've researched and can't find a clear way to do this on either side.
How do I limit how much the database sends at one time to the Django Rest Framework and/ or The Dojo Dgrid. The Dgrid pulls the data from the Django Rest API. The DRF pulls data directly from the MySQL database.
If I can control how much data is sent at one time, then hopefully it won't lock up the browser as much. ANY suggestions, advice, help, examples would be very helpful. Thanks in advance!
UPDATED 6.22.15 -
Alright, I FINALLY Got the pagination to work and it display the limit/offset in the headers. :) YAY!!!! I can also see the data in the Response headers. HOWEVER... the grid won't populate and I keep getting this odd error:
TypeError: transform(...) is null
return transform(value, key).toString();
instrum...tion.js (line 20)
I've gotten this error before, but I've never been able to find a solution to it. After researching, there's not much I can find on HOW to fix or really even what it is. Any help with this would be greatly appreciated!! I'm SO CLOSE to getting this thing to work correctly after WEEKS and WEEKS of beating my head against a wall. Please help! :) Thanks in advance!!!
2nd Update - This was an answer from a previous post - but I'm still not sure how to fix it. When I addressed another issue - it went away for awhile, but I still have no idea how to correct the issue.
Problem 3: "transform(...) is null return transform(value, key).toString();"
This sounds largely tangential to the original issue, but the most common cause is a widget template that is referencing a property via ${...} that doesn't actually exist in the widget.
I don't know how to answer this on the layer between DRF and the database, but as discussed in other SO questions like this one, DRF allows you to limit the amount of data sent with requests via page or offset/limit parameters.
Based on the phrasing of your question, it sounds like the client side is actually requesting too much data. I'll outline how the flow should work, so hopefully you can spot what you've missed:
A dgrid instance is set up with a collection referencing a dstore/Rest instance
The dstore/Rest instance is created with appropriate properties set. In this case, based on the DRF Documentation:
useRangeHeaders: false (this is already the default)
rangeStartParam: 'offset'
rangeCountParam: 'limit'
As a result, when the grid renders, you should see requests sent to your server e.g. endpoint?offset=0&limit=25 - if you don't see those two parameters, that would be why you're getting too much data
The server will need to query the database with the respective OFFSET and LIMIT
The server must provide a response with the expected number of items (except if it reaches the end of the data set first, which should be reflected by the total property in the response, presuming the customization in the previous SO answer I linked is used)
Ultimately, if the service is working as expected, the grid should only be requesting a handful of items at a time, and should only be firing one or two requests at any given time.
Would add as a comment, but not enough reputation at the moment ....
Your question is pretty general, but one strategy would be to allow the user to select the number of items they wish to view at a time and then allow the user to page through the data with 'next x items' and 'prev x items' buttons. Your data object query would then use the current position +/- 'x' as the index range to reduce the number of data objects sent to the browser. This is the basic flow for Ebay, Amazon, Google, or any site with thousands of items to display. The 'next' and 'prev' actions could be wired as POST requests.
Does anybody have any experience with as3-spod?
I downloaded the source code from github and as3-signals and started to try it out, but I´ll take ages to get to know the framework by trial and error and probably miss a lot of best practices. The framework looks good but lack's on examples. The git page does't have a lot of info on that...
If anybody knows some other ORM for AIR that I can use on pure AS3 projects that have any bit of documentation, I´m more than thankful!
I was hoping to do a question-comment asking for clarification, but I don't have enough reputation yet! So I will answer as best I may.
I am using as3-spod for my application. It's been pretty reliable and mostly given me what I want. It's not really ideal, though. What I'd really like is something more ActiveRecord-like, or something original that lets you generate queries by concatenating conditions in a fluid syntax.
But if you're not using Flex (as I'm not, and you're not) then your options are pretty thin, as most of the other AS3 ORMs out there rely on some part of the Flex framework. Apart from as3-spod, the only possibility I could find was Christophe Coenraets' proof-of-concept but as he points out, it would need a lot of work to develop it into a fully-fledged ORM:
This is still a simplistic proof of concept and is by no means a production ready ORM solution.
And I haven't had time for that.
You are right that as3-spod is quite poorly documented. I guess the main class you want to look at is SpodTable. It's from that one you do inserts, selects, etc. An update on a single object can be done from the object itself. Look out for the various signals on SpodTable (select, selectAll, etc). To get going with it, just mark up a model class with metadata, then from your SpodDatabase instance call createTable(MyModelClass).
My main gripes with as3-spod are these (I'm listing them so you don't look for features that don't exist, which I wasted a fair bit of time doing!):
It works asynchronously. Doesn't matter if your actual SQLConnection has been opened synchronously or asynchronously; you have to listen to signals. That means you can't retrieve records and then use them straight away in the same method, you have to listen to signals. What I tend to do is to do large selects when the app starts, then filter the data in memory rather than doing complex queries. Pretty annoying.
Be careful with null values for numeric columns. I can't see a way of setting NULL or NOT NULL for columns using as3-spod; it always seems to make them NOT NULL, which will cause errors if you try to insert a row from an object with null fields.
There's no migration system (a la Rails). I am working on rolling my own as that's an essential feature for my purposes (it's a mobile app I'm developing).
Good luck! Let me know in comments if there's anything else specific you'd like me to cover and I can expand this answer.
EDIT
I've just noticed the existence of AS3SQLite. Haven't used it yet but, looks like there are other possibilities out there :)
I have been thinking of trying out component based architecture for game development. I have read some blog posts and articles about it but I have a few things I have not sorted out yet. When I say component based I mean that you can add components to a ComponentManager that updates all components in its list. I want to try this to avoid getting classes that inherits variables and functions that they don´t need. I really like the idea of having an really simple entity class with a lot of components working side by side without getting bloated. Also, it is easy to remove and add functionality at runtime wich makes it really cool.
This is what I am aiming for.
// this is what the setup could be like
entity.componentManager.add(new RigidBody(3.0, 12.0));
entity.componentManager.add(new CrazyMagneticForce(3.0));
entity.componentManager.add(new DrunkAffection(42.0, 3.0));
// the game loop updates the component manager which updates all components
entity.componentManager.update(deltaTime);
Communication: Some of the components need to communicate with other components
I can´t rely on the components to be self-sustaining. Sometime they will need to communicate with the other components. How do I solve this? In Unity 3D, you can access the components using GetComponent().
I was thinking of doing it like this, but what happens if you have two components of the same type? Maybe you get a Vector back.
var someComponent : RigidBody = _componentManager.getComponent(RigidBody);
Priority: Some components need to update before others
Some components need to update before others to get the correct data for the current game loop. I am thinking of adding a PRIORITY to each component, but I am not sure that this is enough. Unity uses LateUpdate and Update but maybe there is a way to get an even better control of the order of execution.
Well, thats it. If you have any thoughts don´t hesitate to leave a comment. Or, if you have any good articles or blogs about this I will gladly take a look at them. Thanks.
Component based game engine design
http://cowboyprogramming.com/2007/01/05/evolve-your-heirachy/
EDIT:
My question is really how to solve the issue with priorities and communication between the components.
I settled for an RDMBS component entity system http://entity-systems.wikidot.com/rdbms-with-code-in-systems#objc
It is so far working really well. Each component only holds data and has no methods. Then I have subsystems that process the components and do the actual work. Sometimes a subsystem needs to talk to another and for that I use a service locator pattern http://gameprogrammingpatterns.com/service-locator.html
As for priorities. Each system is processed in my main game loop and so it is just a matter of which gets processed first. So for mine I process control, then physics, then camera and finally render.