Redirect outgoing external asset requests to local filesystem with NodeJS - html

Let's say we have a request to an S3 bucket to get an image:
<img src="https://s3-us-west-2.amazonaws.com/{BUCKET}/logo.png" />
I need to work on this project without having access to the internet, so within my Express server, I need to find a way to redirect all requests from https://s3-us-west-2.amazonaws.com/{BUCKET} to ~/Desktop/project/{BUCKET}.
Is there a way to do this via proxying, or would it be a better idea to cut a new branch and replace all external asset links with local file locations?

You would have something get something like this in your network panel
"https://s3-us-west-2.amazonaws.com/{BUCKET}/logo.png"
You can basically remove all "http://s3-us-west-2.amazonaws.com"
And lets say you run it on localhost:3000, your request will look like http://localhost:3000/{BUCKET}/logo.png
You can add following lines in your express server.
var request = require('request');
var proxy = true //if running locally else false
app.get('/{BUCKET}/logo.png', function (req,res) {
if (proxy)
res.sendFile('/home/Desktop/project/' + req.url)
else {
var options = {url : 'http://s3-us-west-2.amazonaws.com' +req.url,
method: 'GET'};
req.pipe(request(options)).pipe(res);
}
)}
The problem with this may be that for every asset it requests from s3 it will always go through your express server and the load of getting the assets will come on express server. You can do it for development but it is not recommended for production.
So for the final deploy you can put all the "http://s3-us-west-2.amazonaws.com" again.
If you don't want to do it programmatically, you can use proxy tools like charles or fiddler. They capture all the traffic from your system. You can create rules for particular requests or set of requests to fetch from local instead of s3.

Related

Connecting Database with Svelte

I'm new to using Svelte and would like to create a ordering website using Svelte. I know that I will need a database to keep track of the order, customer name, price etc. I have used MySQL before but I haven't learned how to connect a database to a website.
Is there a specific database that you can use if you are using Svelte?
Or is there a way to connect MySQL to Svelte?
I have searched about this on Youtube and Google but I'm not sure if it's different if you are using Svelte so I wanted to make sure.
Note: I have not started this project yet so I do not have any code to show I just want to know how you can connect a database if you're using Svelte.
Svelte is a front end javascript framework that run on the browser.
Traditionally, in order to use databases like mysql from a front end project such as svelte, (that contains only html,css and js), you would have to do it with a separate backend project. You can then communicate the svelte app and the backend project with the help of REST api. The same applies to other other front end libraries/frameworks like react, angular vue etc.
There are still so many ways to achieve the result. Since you are focusing on Svelte here are few things options
1 Sapper
Sapper is an application framework powered by svelte. You can also write backend code using express or polka so that you can connect to database of your choice (mysql / mongodb)
2 User Server less database
If you want you app simple and just focus on svelte app, you can use cloud based databases such as firebase. Svelte can directly talk to them via their javascript SDK.
3 monolithic architecture
To connect with mysql in the backend, you would need to use one serverside application programming language such as nodejs (express) php or python or whatever you are familiar with. Then use can embed svelte app or use api to pass data to the svelte app.
I can make an example with mongodb
You have to install the library
npm install mongodb
or add in package.json
Then you have to make a connection file that you have to call everytime you need to use the db
const mongo = require("mongodb");
let client = null;
let db = null;
export async function init() {
if(!client) {
client = await mongo.MongoClient.connect("mongodb://localhost");
db = client.db("name-of-your-db");
}
return { client, db }
}
for a complete example with insert you can see this video
https://www.youtube.com/watch?v=Mey2KZDog_A
You can use pouchdb, which gives you direct access to the indexedDB in the browser. No backend needed for this.
The client-pouchdb can then be replicated/synced with a remote couchdb. This can all be done inside you svelte-app from the client-side.
It is pretty easy to setup.
var db = new PouchDB('dbname');
db.put({
_id: 'dave#gmail.com',
name: 'David',
age: 69
});
db.changes().on('change', function() {
console.log('Ch-Ch-Changes');
});
db.replicate.to('http://example.com/mydb');
more on pouchdb.com
Also the client can save the data offline first and later connect to a remote database.
As i get question mostly about connection to backend, not a database. It is pity, but svelte app template has no way to connect backend "in box".
What about me, i'm using express middleware in front of rollup server. In this case you able to proxy some requests to backend server. Check code below
const proxy = require('express-http-proxy');
const app = require('express')();
app.use('/data/', proxy(
'http://backend/data',
{
proxyReqPathResolver: req => {
return '/data'+ req.url;
}
}
)
);
app.use('/', proxy('http://127.0.0.1:5000'));
app.listen(5001);
This script opend 5001 port where you have /data/ url proxied to backend server. And 5000 port still available from rollup server. So at http://localhost:5001/ you have svelte intance, connected to backend vi /data/ url, here you can send requests for fetching some data from database.

HTML server sent events

I have a little understanding on how server sent events work. Say I have a Linux server(remote server) and I need to monitor it's CPU usage from local machine continuously (via a HTML page which will be in my local machine). Will I be able to get the CPU usage continuously from the server to local machine using SSE? If so, I need some clarifications on how to do so. Or is there any other alternatives that I can go with without involving any softwares or so?
You'll need to write some code to run on the server, which will gather whatever data you need. (Common choices include Node.js, PHP, etc.)
That code will need to either directly serve HTTP requests, or connect to a web server.
Your code will send data in this format:
event: someevent
data: {"key": "value"}
Then, your client-side will use EventSource:
const eventSource = new EventSource('https://example.com/your-sse-path');
eventSource.addEventListener('someevent', (e) => {
console.log(JSON.parse(e.data));
});

Setting up local graphhopper server

I am trying to get a simple graphhopper client to retrieve route response from my own hosted graphhopper server, but I am not able to retrieve any data from the local service. I've setup a local graphhopper service by following this guide: https://github.com/graphhopper/graphhopper/blob/master/docs/core/quickstart-from-source.md
I've downloaded the OSM map NewYork.osm.pbf for working with a new york map data. After cloning the project and checking out the master branch, I start the service by running the following:
./graphhopper.sh web NewYork.osm.pbf
On the client side I am following this guide: https://github.com/graphhopper/directions-api-java-client
After setting up the Maven dependency on the client project, my client tries to perform a very simple route retrieval of two valid New York City coordinates: (Chelsie Piers and Columbus Circle)
GraphHopperWeb gh = new GraphHopperWeb("localhost:8989/api/1/route");
gh.setDownloader(new OkHttpClient.Builder().
connectTimeout(5, TimeUnit.SECONDS).
readTimeout(5, TimeUnit.SECONDS).build());
// specify at least two coordinates
GHRequest req = new GHRequest();
req.addPoint(new GHPoint( 40.7469, -74.0083)).addPoint(new GHPoint( 40.7681, -73.9824));
req.setVehicle("foot");
req.setLocale(Locale.US);
GHResponse fullRes = gh.route(req);
if(fullRes.hasErrors()) {
return;
}
currently fullRes response object's error member is valid: no paths, main errors: [java.lang.RuntimeException: Not found]
If I run the same example on the graphhopper hosted server by setting the client to :
GraphHopperWeb gh = new GraphHopperWeb();
gh.setKey("my graphhoper API key");
this retrieves a route response just fine. I am guessing I am not setting certain configuration files properly. I would love to know how to get this example working as well as know how to configure the local servers to generate API keys for graphhopper clients that need to authenticate to locally hosted graphhoper servers. Any help or links would be much appreciated.
Thank You.

Web API call not returning

I have a RESTful Web API that is running properly as I can test it with Fiddler. I see calls going through, I see responses coming back.
I am developing a tablet application that needs to use the Web API in order to fetch data or make updates in the repository.
My calls do not return and there is not a single trace in the Fiddler to show that my calls even reach the server.
The first call I need to make is to login. The URI would be this:
http://localhost:53060/api/user
This call would normally return some information about the user (such as group membership, level of authorization and so on). The Web API uses Windows Authentication, so the repository is able to resolve all these fields based on the credentials passed in. As I said, in Fiddler I see the three calls made to the URI as the authentication is negotiated between the caller and the server. The third call returns with a JSON object that contains all information generated from the repository as expected.
Now, moving to my client I have the following:
var webApiClient = new HttpClient(new HttpClientHandler()
{
UseDefaultCredentials = true
})
{
BaseAddress = new Uri("http://localhost:53060/")
};
webApiClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
HttpResponseMessage response = await webApiClient.GetAsync("api/user");
var userLoginInfo = await response.Content.ReadAsAsync<UserLoginInformation>();
My call to "GetAsync" never returns and, like I said, I see no trace of it in Fiddler.
Any idea of what I'm doing wrong?
Changing the URL where the Web API was exposed seemed to have fixed the problem. Thanks to #Nkosi for the suggestion.
For anyone stumbling onto this question and asking themselves how to change the URL of the Web API, there are two ways. If the simulator is running on the same machine with the Web API, the change has to be made in the "applicationhost.config" file for IIS Express. You can locate this file by right-clicking on the IIS Express icon in the Notification Area (the bottom right corner) and selecting show all websites. Highlight the desired Web API and it will show where the application host configuration file is located. In there, one needs to locate the following section:
<bindings>
<binding protocol="http" bindingInformation="*:53060:localhost" />
</bindings>
and replace the "localhost" name with the IP address of the machine where the Web API is running.
However, this approach will not work once you start testing your tablet app with a real device. IIS Express must be coerced into exposing the Web API to the outside world. I found an excellent node.js package that can help with that. It is called IISExpress-proxy.

node.js - repeatedly updating webpage from mysql database

I am trying to create a node.js app to automatically update a webpage every few seconds with new data from a mysql database. I have followed the information on this site: http://www.gianlucaguarini.com/blog/push-notification-server-streaming-on-a-mysql-database/
The code on this site does indeed work, but upon further testing it keeps running the "handler" function and therefore executing the readFile function for each row of the database processed.
I am in the process of learning node.js, but cannot understand why the handler function keeps getting called. I would only like it to get called once per connection. Constantly reading the index.html file like this seems very ineffecient.
The reason that I know the handler function keeps getting called is that I placed a console.log("Hello"); statement in the handler function and it keeps outputting that line to the console.
Do you provide the image URLs that the client.html is looking for? Here's what I think is happening:
The client connects to your server via Socket.IO and retrieves the user information (user_name, user_description, and user_img). The client then immediately tries to load an image using the user_img URL. The author's server code however, doesn't appear to support serving these pictures. Instead it just returns the same client.html file for every request. This would be why it appears to be calling handler over and over again - it's trying to load a picture for every user.
I would recommend using the express module in node to serve static files instead of trying to do it by hand. Your code would look something like this:
var app = require('express')();
var http = require('http').Server(app);
var io = require('socket.io')(http);
http.use(app.static(__dirname + "/public"));
That essentially says to serve any static files they request from the public folder. In that folder you will put client.html as well as the user photos.