I am very new to database design and structuring - I have had no formal training and am purely self taught so I apologize in advance if this is a bland question.
I am designing a web app and am thinking to the future as users will have to be able to interact with each other sharing part of their data. I am wondering if there is a standard convention to controlling access to tables in MySQL and how I should generally tackle this problem with code written in NodeJS, ExpressJS, KnexJS, and BookshelfJS.
For example: a user will be matched with another user, both users will be able to see location, favourite book, etc but not able to see last name, birth date. etc.
How do I control this?
If anyone could point me to a few resources they have found helpful that would be great as well.
You seem to have learned a bit of MySQL and its access control features. Well, database user level access control IS NOT used by modern applications -- that could make resource management, like connection pools, very hard to implement. Usually SQL databases backing web applications have a single or, at most, two users: one for general data access and one for admin purposes.
The kind of access control you mentioned MUST be handled by your application code, YOUR code. There are libraries that help take care of authentication (e.g. passport) and authorization but ultimately it is YOUR CODE responsibility.
So my answer to your "How do I control this?" question is:
With YOUR code.
This is the whole point of Software Development.
Related
I have a question regarding the best solution to adopt for a client. My client has an existing Access
database with tables, queries, reports and lots of them! He requires an application/solution that will :
allow it to be used online/cloud.
allow the application users to login and access only their record data.
allow my client to have a main-admin account to login too and adminster all client accounts.
allow the application users to add a text box via a form to add a new field to the table
allow application users to upload documents against a record
Heres my dilemma; as a PHP/Open Source developer i could write an application which does all of this; having already used a 3rd party program to convert his Access dB to MySQL (though i will have to manually convert the Access Queries forms to MySQL Views.
This is my preference since i have total control and confidence with the dev tools i'll use. Down side is it will take absolutely ages because of the number of tables and queries. Alternatively, i recently read about MS Sharepoint which i know nothing about other than what i read. SharePoint looks like it could handle this solution very easily especially the MS Access stuff, Microsoft now
as of 2018 recommend using 'PowerApps' for building Access Apps in sharepoint. It all looks quite do-able but i'm not a MS aficionado and dont want to get in 'out-of-my-depth'.
Having done (and enjoyed) many years of MS VB development up to about 5 years ago, i know Sharepoint will allow you to work at a level which requires less Programming skills and more Power-user skillsets.
Can any one advise which they think would be the best route, im not asking for the finite detail - just a pointer from anyone that uses Sharepoint and knows its capabilities beyond the little i've read. Im looking to invest some time into learning it for future projects as it looks great; but for this project isnt it a little to simple???
Many thanks
Access is a Windows application - and therefore not browser based. There is no port to html technology - it is a complete re-write.
For geographically distance users - as with any Windows application - the available solution is the Remote Desktop technology.
It is / would be unusual for any application to allow the end users to redefine the table structure. Thus the idea of adding new fields to a table is not feasible but perhaps one is using the wrong terminology and intends instead to be adding new records to the table - which would be routine.
The other features are all do-able in Access.
For years we've seen users wanting a magic transform from their Access Windows application to a web based application. It simply does not exist.
At the moment I am working on a CRUD app that I am going to deploy (someday) and use for my own startup company. However I am nowhere near finishing this product and I stumbled upon a question that I can't seem to figure out.
I am using Express to serve angular the data out of my MySQL database. To do this I had to create '/api/' routes. However if I go (for example) to '/api/clients' I will be able to see the entire list of clients in an ugly array. In this case that does not really matter since it's just the data they were able to see anyways.
However my question is, is it important to block these kind of routes from users? Will problems arise when a user goes to 'api/createClient'? Could this result in a DB injection that could ruin my db?
My project can be found here: https://github.com/mickvanhulst/BeheerdersOmgevingSA
The server-side routing code can be found: server > Dao > clientDao.js
Controllers, HTML & client-side routing can be found in the 'public' folder.
I hope my question is clear enough and someone will be able to answer my question. If not, please state why the question is not clear and I will try to clarify.
Thanks!
Looking at the code, it looks like your URLs can directly be accessed using browser and if yes, then this does pose a security concern.
Doing DB transaction with the user provided fields or values is major security concern, if these data are not validated and sanitised before making a database call.
I would recommend following minimum steps to follow before crafting APIs which is internal but can be accessed using browser -
If this is internal, then do not provide HEADER ACCESS CONTROL from the server or keep it confined only to your domain name. This prevents any ajax call to be made to your APIs from another domains.
Do sanitise and validate all the data thoroughly before doing any kind of database transactions. There are lots of material on this everywhere on how to do it.
If these APIs are meant to be used for internal purpose, then kindly provide some kind of authentication to your APIs before doing the logical work in your routes with the help of middle-wares. You can leverage cookie authentication for very simple API authentication management. You can also use JSON Web Tokens, if you want a more levels of security.
If you are manipulating your databases then I would highly recommend to use some kind of authentication in your APIs. Ofcourse, point number 2 is must.
I was hoping someone could help me answer a couple of questions regarding Tableau. I am not as familiar with the platform, but I have a client who is looking for a reporting/analytics/data visualization platform that they could use for many of the internal apps (for their employees) and external (customer facing internet with login) applications.
The driver is that each of their internal teams has used many disparate technologies such as SSRS, Crystal, custom ASP.NET controls (Kendo/Telerik, etc), but now they have the opportunity to choose a common platform that could serve most/all of the future reporting and data visualization needs for enterprise and customer facing solutions.
They are looking for a platform that provides everything from simple grids with basic filter/sort/group, all the way to rich charting and ad-hoc reporting with slicing and dicing of data.
They will not always be creating dashboards in these apps since they are customer-facing, but they may want to have dashboards for internal (intranet) apps. They will definitely want the ability to build true internal BI dashboards to report on data from all these online apps across all customers, to whom they provide their SaaS/customer-facing web apps.
One of our main concerns revolves around security of data, as some of these customer-facing web apps are multi-tenant, so we'd need to ensure that data is always filtered by the client tenant id. Also we have a very customized security model, with data driven roles, permissions that may prevent showing certain types of data (e.g. SSN, Salary, etc) etc.
Does Tableau fits this model, can it meet most/all of these requirements, or is it meant more for internal data?
It should be quite possible by setting up a reverse proxy that would front end your multi tenant web application. There is a document on how to setup Apache as reverse proxy with Tableau with/without SSL.
I am familiar with how to configure Apache as reverse proxy and so here are the details with Apache Web server on how to setup reverse proxy rules.
There may be some documentation for front ending with IIS/Nginx so you should do some googling by yourself.
You need to harden your webserver configuration by limiting access from the external firewall to read only pages and the internal user can access allpages. Since you mentioned that the external users are allowed access to readonly pages, I presume all the requests from external requests will be only GET requests and a few PUT/POST requests when users choose to use filters. So you can block external users from any request except GET. Exceptions should be made for the pages that allow applying filters and grouping.
In your mutitenant application make sure you refer to the tableau URL's by the apache server url that is exposed to the outside world. If any url not configured in apache is used, users will recieve a access denied error. You need to create a role that has readonly access to tableau pages for external users. To address mulitenancy you need to set a cookie or something to identify the tenant and something similar to identify the user. To filter SSN and some more information you can use mod_proxy_html which filters content. You can also use mod_security module of Apache to block SSNs and Credit Card Numbers.
References:
Configuring Apache Server as Proxy with Tableau
Apache mod Proxy documentation
Blocking POST requests
mod_security FAQs
Yes to most of your questions -- with just a little fine print.
First remember Tableau is primarily about visualizing data, so it is great for publishing readonly interactive views of data. If you want allow end users to edit data, you'll have to do that by another means. Fortunately, the Tableau JavaScript API lets you interact closely with Tableau with your custom Javascript code. So if your needs are mostly about visualization, but want want to be able to trigger some custom code to modify data in some of your apps, you should be fine. But Tableau is not designed for creating custom CRUD apps as a rule.
The great thing about Tableau server is that many people can learn to use it and publish their own visualizations -- even if they don't know how to program. That doesn't mean they will win visualization design awards the first time, or that they shouldn't learn something about how databases work if they want have good performance. But it does mean the people that know their data best can learn to design and publish their own visualizations without having to wait three months on a backlog queue so the one IT guy can change the color of a button or add a field. It still would be good to get good system, database and visualization folks to help train, organize data, set governance and security rules, optimize, etc, but business users can learn to be the ones with hands on control over how their information is presented. That's a good thing.
The security question has several moving parts, and usually there are usually good answers from Tableau depending on what you're trying to accomplish. Tableau server does support multi-tenancy using sites. There is fairly flexible permissions and group policy system. It can use SAML for authentication, and has several features providing access to specific to the user/tenant. It works with almost every database, and you can in some cases push your security enforcement to the database server -- SQL server for instance. There is a trusted ticket feature where you can defer some authorization decisions to another server, say a web portal server. Useful when Tableau visualizations are embedded in some other web page.
Most security use cases can be supported out of the box, but there are some complex custom access control situations that are tricky to implement currently in Tableau server. Nothing you've listed sounds out of the normal swim lane, but the only way to know whether your security model is too complex is to dive into the details. Hopefully they will release a custom access control API for users who want to extend it.
At the high level, you sure can use Tableau to build customer-facing dashboards. You can quickly build and deploy those and as others mentioned, you can iFrame them with Javascript APIs, you can customize most of it. But it doesn't provide complete flexibility for user interaction, which you can if you use other technologies. Other options include hand coding framework and then using charting applications.
For simple dashboards, Tableau would be the obvious choice if you have already bought core-licenses. But when looking at what's going on in the industry, Tableau will not be able to fulfill all needs.
If using Tableau
1. Building Charts/Tables/Visualization is a super simple, efficient way.
2. You can expose low grained data to customers, because of Tableau's propitiatory columnar database engine, you can potentially expose millions of records via a dashboard.
3. You can use Tableau's security and access control mechanism.
4. As other user mentioned, you can use trusted ticketing mechanism to integrate easily with other applications (portals etc).
Challenges with Tableau approach.
1. If you have late arriving transactions (in Internet world it's so common to mark a click as fraudulent after few days) with late arriving transactions, you have to have full refresh the extracts, which means if you are showing say 13 months worth of data, you have refresh it all, all the time. Now with bigData, business needs all data all the time, which means you would end up extracting millions of records, throughout the day.
2. Very little flexibility in user interactions, like menus,drop downs etc. you have to work with what's been provided by Tableau.
3. If you have multiple charts on same dashboard page, not so user friendly way to download underlying data.
4. Many other challenges, in laying out visualizations on dashboard page, as there is no easy way to control canvas with pixel control, white spaces etc.
You should be very careful, after analyzing your use case, whether Tableau would be the right product before you invest in it.
Tableau's primary power comes from its desktop tool for data visualization/exploration and not from pre-built dashboards.
Best of luck.
Since Tableau public is also based on Tableau, I assume that you can put your dashboards in public using your own Tableau infrastructure.
I would like to design a database application using Microsoft Access. Before I start there are some important features I want to make sure are available in Access.
In a multi-user environment can the database be accessed simultaneously by different users such that only individual records are locked/unlocked as necessary?
Does Access need to be "opened" or can a "front end" be designed so the user only sees menus, menu bars, tabs, data screens, etc?
Can the database design features be locked so the user cannot change any database features?
Thank you for your help.
AF
Yes, certain records can be locked.
For Example, if you work with an ADO Recordset:
recordset.Open Source, ActiveConnection, CursorType, LockType, Options
LockType defines the locking-mechanism used for the selected Query Source.
More Information on that: http://msdn.microsoft.com/en-us/library/ms675544%28v=vs.85%29.aspx
Usually you split your Database into an frontend and backend. The back-end only contains only the tables, the rest goes in to the frontend. The backend tables are then linked to the frontend. Here is a link explaining how to do that: http://www.fmsinc.com/MicrosoftAccess/DatabaseSplitter/
In the frontend file can be compiled, so that the frontend users cannot edit sources.
In response to the third of your initial questions please note that Microsoft Access offers the facility to convert an Access database to an executable file (having file extension "accde" where your forms, reports, code and macros are protected to a substantial extent. However the level of protection offered with regard to tables is low in that the same are directly accessible even in an accde. MS Access also offers the runtime version free of charge and does not restrict its deployment to as many users as you may require. It no longer offers user-level security. This feature has been replaced with an encrypted password feature. Thus you will have to insert your own design and code to impose access restrictions on different categories of users. Alternatively, to use the user level security feature, one has to develop his application in an earlier version of Access in which this feature was available and thereafter use a recent version to convert it. I am uncertain whether MS Access 2013 will entertain such a database but understand that MS 2010 will though it will not offer that feature when you create a new database inside it.
You can get SQL Express for free, it's a stripped down version of SQL Server. It will handle multiple user access significantly better and not come with the bloat/performance degredation issues that Access brings along.
There will be a slightly higher learning curve, but if you're looking to get your foot into the development world's door, you'll be much better off learning how to work with SQL Server than Access.
I have been asked by a friend to help him assess a number of quotes for porting a desktop application based on MS access and VBA to a web based app. The application seems to have a relatively large amount of business logic coded into the VBA.
My question is very specific - are there any good tools or resources out there which could assist the porting from access, rather than doing a complete re-write?
The end technology used for the web app does not matter hugely, but would ideally be as mainstream as possible.
You may explore the possibilities offered by Sharepoint. It may help you get the data accessible online but how well will that work depends also on how much VBA code was used in the Access application.
There are some tools around that pretend they can convert MS Access to PHP/ASP websites like DB Forms, but I haven't tried them and they usually only convert the visible part of the app and not the queries and VBA.
They can be helpful to get started though.
Random thoughts
The VBA tends to be the biggest issue.
Moving to ASP.Net will take time and for that you are faced with difficult choices:
transfer all code to the ASP.NET to just get it working
rethink the structure and do a proper ASP.Net implementation from scratch.
I'd prefer the first one: just try has hard as possible to get results fast.
Use SSMA to move the data to SQL Server (unless you want to keep Access as the backend).
Make the forms look the same as on your existing application (or at least have the same function), port the VBA to VB.Net (or C# if you feel like it) form by form, module by module and test that they work as you go along.
Don't try to refactor or make things better at this stage, the point is to 'slap' the old code on the new 'system' and make it bark as it used to, not better, not worse.
Only then can you start refactoring and improving using the new tools at your disposal.
I'm saying all this assuming that there was nothing terribly wrong with the old app and that it just needed to be ported for online consumption.
If the old app was defective and wasn't fulfilling its role, then more emphasis should be placed on re-thinking which parts should be translated and which one should be reworked.
At any rate, you need to have a detailed action plan and a review of the current code and functionalities and try to limit as much as possible your expectations for the first version of the new system: avoid letting everyone input their wishes or your project will become horrendously difficult.
Concentrate on the minimum needed to achieve a certain level of functionality that will satisfy your users, then build on that.
There may be some tools to some of the basic stuff, like to upsize to a different database or maybe the look and text boxes of the forms, but converting what sounds like a lot of VBA code, not so sure.
Is this an intranet/local network type of web app or are you putting it out on the internet? Security will become a major difference between this and your Access app.
Make sure they understand Access/VBA so you can maintain the business logic that has been over the life of the Access app.
Convince your friend to stop/slow any development on the Access app to prevent the company from aiming at a moving target. This may not be realistic, but really needs to be considered.
Is there a reason why hosting the app on Windows Terminal Server would not suffice? This means zero changes to the app, no reprogramming cost and no danger of losing crucial business logic. If you use the Citrix extensions, you can run it in a web browser (though I guess that only works with IE -- I've never used them). But the RDP client comes in versions for Mac and Linux as well as Windows, so you can basically support anybody as long as they install the RDP client for their OS.
Yes, it's more installation on the client end, but it's a helluva lot cheaper and easier on the development and avoids the problem of losing important things coded into the Access app.
Of course, supporting large user populations on WTS/Citrix can get expensive and if the Access app is in need of re-engineering, anyway, it can change the balance. But it's something that you should consider. It's really easy to set up WTS, in fact, and provisioning a server for it basically a matter of adding RAM and Internet bandwidth (though RDP is really efficient to begin with).
One key mistake many people make when trying to run an Access app on WTS:
YOU MUST SPLIT THE DATABASE (front with forms/reports/etc., back end with data tables only), and each user must have their own copy of the front end (stored in user profile on the WTS, or in a folder on your WTS server's data partition with appropriate permissions assigned to the user groups authorized to use the app). Tony Toews's front-end updater is very useful in this context, and explicitly engineered to work in a Terminal Server environment.