I am currently developing a full-stack web application for the first time. It is a store that needs to give users the ability to upload "books", edit, delete, and manage them. As of now, I have a React front-end, that calls an Express API using Axios that queries a MySQL database. This currently can manage the product details, titles, and simple labels and relations.
However, I now need to store images and .json files dynamically as well. So, I have researched and need to use Azure Storage to store these images and allow access to them by the end-user. I have researched it and the client would like to use Azure storage as well.
I have gotten quite overwhelmed looking into the Azure documentation for Javascript image uploading, and every "tutorial" starts with create a web application, storage account, and app service.
All I would like to do is store images from the user in Azure storage, so that when I eventually deploy the website, the data is available to be accessed and then my front end or API can call Azure to get the images for the user. I apologize if the question is confusing or vague, I am really just overwhelmed by Azure's documentation and it seems like such a simple and common problem. Any guidance or references would be greatly appreciated.
Yes, it's quite possible. You simply create an Azure Storage account and upload your files as blobs via that account. Then they will be available publicly on the Internet, and your web application can reference them from wherever it is hosted.
It is possible as I believe it is quite a common practice to have a Content Delivery Network to deliver the images. I am not very familiar with Azure but I am with AWS and I can tell you that you can use an AWS S3 bucket to store the images and JSON. It comes with many different configuration options to allow content to be protected or open to the general internet.
Related
Is it possible to use two databases, example, Firebase and an online webhosting server? I am developing an app that uses the webhosting server for the data and the firebase only for the chats.
Absolutely. Firebase is very flexible when it comes to picking and choosing parts which you want to use.
You don't need to migrate away from whatever backend you already have if you just need to add in something that is unrelated to your site data, such as a support chat or something.
Instead, see it as an opportunity to discover Firebase by trying out something isolated like a chat using Firebase Database. You'll find your best fit by trying out the strengths and limits of the platform.
Yes it is Possible but, you have to manage the back-end accordingly. I have created the app in which api is hosted in aws-EC2 and few part is in the Firebase. For example for some live changes I'm using the Firebase and for other things like getting feeds and all those thing i used aws EC2.
I am working on a social network app from scratch,
I designed my database (choosing Mysql) to store and retrieve data
The problem starts when I am posting videos and photos and when data become larger and larger when I am waiting to view news feeds (depending on sql query that brings the last posts from friends ans Pages).
The question is :
How can I manage and handle the big data, and how can I make the news feeds service more efficient ?
What do you think of using (Real-time) Databases from Firebase?
If you're thinking about putting up Firebase as your choice for your database, then there's nothing wrong with that. Firebase realtime database is a very secure and great option.
Firebase gives you functionality like analytics, databases, messaging and crash reporting so you can move quickly and focus on your users.
Firebase is built on Google infrastructure and scales automatically, for even the largest apps.
And on top of that, many great apps do use Firebase as their backend.
I'd say to know more, you can just start with a demo project on Firebase to learn how the things happen with Firebase, and after learning, start using it as backend for your social app.
Just go to Firebase Console and start with your own project. To learn how to use Firebase, do refer Firebase docs and if stuck, StackOverflow is always there.
If you want to get some inner details, I have some repositories on GitHub, that might help you in understanding about Firebase and the security rules and other important things.
Demo App
Firebase Security Rules
I am creating a mobile application using swift for my organization. The application reads in data in JSON format to populate the information that gets displayed on the application. I already have a method to generate the JSON files, but I need somewhere to host the actual files. I have an AWS account and an instance running, this is where I initially was hosting my JSON files but I got an email from AWS saying that having the app constantly grab the JSON files that I stored on the site resembled scanning behaviour, which is not allowed apparently. So I was wondering where I could host JSON files so that my mobile app can read in the information it needs. The biggest thing that I need is that I can host it with a static URL that I can keep calling with my app.
I was thinking of potentially putting the files on an AWS bucket with read permissions and having those get accessed, but since AWS already complained about me doing something like that I'm iffy. I was also thinking of putting the JSON files on Github, but again I'd hate to get an email from github telling me that they don't like that an application keeps grabbing the data.
For background, the app essentially has a hardcoded URL that grabs the JSON data and parses it. I didn't do an api because an API takes some time to grab all the information that doesn't really change that often, it's much easier to generate the JSON files locally and just post them online somewhere. The information on it can be read by anyone too it's not private or anything.
Message from AWS:
Hello,
We've received a report(s) that your AWS resource(s)
information
has been implicated in activity which resembles scanning remote hosts on the internet for security vulnerabilities. Activity of this nature is forbidden in the AWS Acceptable Use Policy (https://aws.amazon.com/aup/). We've included the original report below for your review.
Please take action to stop the reported activity and reply directly to this email with details of the corrective actions you have taken. If you do not consider the activity described in these reports to be abusive, please reply to this email with details of your use case.
If you're unaware of this activity, it's possible that your environment has been compromised by an external attacker, or a vulnerability is allowing your machine to be used in a way that it was not intended.
We are unable to assist you with troubleshooting or technical inquiries. However, for guidance on securing your instance, we recommend reviewing the following resources:
I'm new so it won't let me post links but they attached a couple help links
If you require further assistance with this matter, you can take advantage of our developer forums:
more links I can't have
Or, if you are subscribed to a Premium Support package, you may reach out for one-on-one assistance here:
link
Please remember that you are responsible for ensuring that your instances and all applications are properly secured. If you require any further information to assist you in identifying or rectifying this issue, please let us know in a direct reply to this message.
Regards,
AWS Abuse
Abuse Case Number:
Using an AWS EC2 instance to host static files (which is what it sounds like you were doing?) is pretty standard and I suspect that this is not what Amazon is complaining about. More likely, your instance has been infected by some sort of software which is causing it to request many files from other random servers on the web ("scanning for remote vulnerabilities"). You should check that you have not accidentally publicly posted your AWS credentials (in any form), and consider wiping the instance and resetting it. And of course reply to the email explaining this to AWS.
I know this sounds crazy, but I had a thought and I was willing to try it out. I use GitLab pages for all my online projects, but a lot of them are ASP.NET MVC, which is an issue as I don't think you can run ASP.NET MVC sites on GitLab pages. I then thought, what if I make a site using something like angular or node.js, and have a central API for all my web projects? I thought that was a great idea, until I realized I couldn't use a database either. I guess what I'm asking is, would it be possible to create a REST API that uses JSON files for storage and node.js as the request pages, to create an API without a database?
Of course.
If you think about a database from the perspective of your application code, it is basically just a place to store and retrieve data.
Imagine the database library you are using has two simple methods, store and retrieve. In your application code, you could write db.store('here is the item') and the later on, db.retrieve().
However, those store and retrieve methods could be implemented in many different ways to provide the same effective behavior from the perspective of your application. Some examples:
Send/query the data to/from an external data store, such as PostgreSQL
Write it to a file on disk and read it back later
Store the data in memory
Make HTTP requests to an external system to store the data
Some of these options will be more or less appropriate depending on your exact requirements, however, the general idea is that given a database API, you could implement the exact same method signatures with a completely different approach.
I was hoping someone could help me answer a couple of questions regarding Tableau. I am not as familiar with the platform, but I have a client who is looking for a reporting/analytics/data visualization platform that they could use for many of the internal apps (for their employees) and external (customer facing internet with login) applications.
The driver is that each of their internal teams has used many disparate technologies such as SSRS, Crystal, custom ASP.NET controls (Kendo/Telerik, etc), but now they have the opportunity to choose a common platform that could serve most/all of the future reporting and data visualization needs for enterprise and customer facing solutions.
They are looking for a platform that provides everything from simple grids with basic filter/sort/group, all the way to rich charting and ad-hoc reporting with slicing and dicing of data.
They will not always be creating dashboards in these apps since they are customer-facing, but they may want to have dashboards for internal (intranet) apps. They will definitely want the ability to build true internal BI dashboards to report on data from all these online apps across all customers, to whom they provide their SaaS/customer-facing web apps.
One of our main concerns revolves around security of data, as some of these customer-facing web apps are multi-tenant, so we'd need to ensure that data is always filtered by the client tenant id. Also we have a very customized security model, with data driven roles, permissions that may prevent showing certain types of data (e.g. SSN, Salary, etc) etc.
Does Tableau fits this model, can it meet most/all of these requirements, or is it meant more for internal data?
It should be quite possible by setting up a reverse proxy that would front end your multi tenant web application. There is a document on how to setup Apache as reverse proxy with Tableau with/without SSL.
I am familiar with how to configure Apache as reverse proxy and so here are the details with Apache Web server on how to setup reverse proxy rules.
There may be some documentation for front ending with IIS/Nginx so you should do some googling by yourself.
You need to harden your webserver configuration by limiting access from the external firewall to read only pages and the internal user can access allpages. Since you mentioned that the external users are allowed access to readonly pages, I presume all the requests from external requests will be only GET requests and a few PUT/POST requests when users choose to use filters. So you can block external users from any request except GET. Exceptions should be made for the pages that allow applying filters and grouping.
In your mutitenant application make sure you refer to the tableau URL's by the apache server url that is exposed to the outside world. If any url not configured in apache is used, users will recieve a access denied error. You need to create a role that has readonly access to tableau pages for external users. To address mulitenancy you need to set a cookie or something to identify the tenant and something similar to identify the user. To filter SSN and some more information you can use mod_proxy_html which filters content. You can also use mod_security module of Apache to block SSNs and Credit Card Numbers.
References:
Configuring Apache Server as Proxy with Tableau
Apache mod Proxy documentation
Blocking POST requests
mod_security FAQs
Yes to most of your questions -- with just a little fine print.
First remember Tableau is primarily about visualizing data, so it is great for publishing readonly interactive views of data. If you want allow end users to edit data, you'll have to do that by another means. Fortunately, the Tableau JavaScript API lets you interact closely with Tableau with your custom Javascript code. So if your needs are mostly about visualization, but want want to be able to trigger some custom code to modify data in some of your apps, you should be fine. But Tableau is not designed for creating custom CRUD apps as a rule.
The great thing about Tableau server is that many people can learn to use it and publish their own visualizations -- even if they don't know how to program. That doesn't mean they will win visualization design awards the first time, or that they shouldn't learn something about how databases work if they want have good performance. But it does mean the people that know their data best can learn to design and publish their own visualizations without having to wait three months on a backlog queue so the one IT guy can change the color of a button or add a field. It still would be good to get good system, database and visualization folks to help train, organize data, set governance and security rules, optimize, etc, but business users can learn to be the ones with hands on control over how their information is presented. That's a good thing.
The security question has several moving parts, and usually there are usually good answers from Tableau depending on what you're trying to accomplish. Tableau server does support multi-tenancy using sites. There is fairly flexible permissions and group policy system. It can use SAML for authentication, and has several features providing access to specific to the user/tenant. It works with almost every database, and you can in some cases push your security enforcement to the database server -- SQL server for instance. There is a trusted ticket feature where you can defer some authorization decisions to another server, say a web portal server. Useful when Tableau visualizations are embedded in some other web page.
Most security use cases can be supported out of the box, but there are some complex custom access control situations that are tricky to implement currently in Tableau server. Nothing you've listed sounds out of the normal swim lane, but the only way to know whether your security model is too complex is to dive into the details. Hopefully they will release a custom access control API for users who want to extend it.
At the high level, you sure can use Tableau to build customer-facing dashboards. You can quickly build and deploy those and as others mentioned, you can iFrame them with Javascript APIs, you can customize most of it. But it doesn't provide complete flexibility for user interaction, which you can if you use other technologies. Other options include hand coding framework and then using charting applications.
For simple dashboards, Tableau would be the obvious choice if you have already bought core-licenses. But when looking at what's going on in the industry, Tableau will not be able to fulfill all needs.
If using Tableau
1. Building Charts/Tables/Visualization is a super simple, efficient way.
2. You can expose low grained data to customers, because of Tableau's propitiatory columnar database engine, you can potentially expose millions of records via a dashboard.
3. You can use Tableau's security and access control mechanism.
4. As other user mentioned, you can use trusted ticketing mechanism to integrate easily with other applications (portals etc).
Challenges with Tableau approach.
1. If you have late arriving transactions (in Internet world it's so common to mark a click as fraudulent after few days) with late arriving transactions, you have to have full refresh the extracts, which means if you are showing say 13 months worth of data, you have refresh it all, all the time. Now with bigData, business needs all data all the time, which means you would end up extracting millions of records, throughout the day.
2. Very little flexibility in user interactions, like menus,drop downs etc. you have to work with what's been provided by Tableau.
3. If you have multiple charts on same dashboard page, not so user friendly way to download underlying data.
4. Many other challenges, in laying out visualizations on dashboard page, as there is no easy way to control canvas with pixel control, white spaces etc.
You should be very careful, after analyzing your use case, whether Tableau would be the right product before you invest in it.
Tableau's primary power comes from its desktop tool for data visualization/exploration and not from pre-built dashboards.
Best of luck.
Since Tableau public is also based on Tableau, I assume that you can put your dashboards in public using your own Tableau infrastructure.