I have a ruby on rails project where I have to make a press page but I don't just want to put the articles in the database (that will take too long) is there anyway that you can pull information from another site and display it on your own site.
This is quite a broad question.
Depending on whether the web applications that you're getting your data from have APIs or not, you would either use those to retrieve the data or you'd have to web scrape it.
For accessing an API you could use Net:HTTP or HTTParty or similar. For web scraping you'd want to use similar libraries for retrieving the page and some parser like Nokogiri to parse the HTML. In both cases you'd want to store the results in your own database to avoid having to scrape/query API upon every request.
Related
I know this sounds crazy, but I had a thought and I was willing to try it out. I use GitLab pages for all my online projects, but a lot of them are ASP.NET MVC, which is an issue as I don't think you can run ASP.NET MVC sites on GitLab pages. I then thought, what if I make a site using something like angular or node.js, and have a central API for all my web projects? I thought that was a great idea, until I realized I couldn't use a database either. I guess what I'm asking is, would it be possible to create a REST API that uses JSON files for storage and node.js as the request pages, to create an API without a database?
Of course.
If you think about a database from the perspective of your application code, it is basically just a place to store and retrieve data.
Imagine the database library you are using has two simple methods, store and retrieve. In your application code, you could write db.store('here is the item') and the later on, db.retrieve().
However, those store and retrieve methods could be implemented in many different ways to provide the same effective behavior from the perspective of your application. Some examples:
Send/query the data to/from an external data store, such as PostgreSQL
Write it to a file on disk and read it back later
Store the data in memory
Make HTTP requests to an external system to store the data
Some of these options will be more or less appropriate depending on your exact requirements, however, the general idea is that given a database API, you could implement the exact same method signatures with a completely different approach.
I'm new to webdev and I'm trying to use passport for registration/authentication on a site I'm setting up. I'm also going to write an application in node later on that will be using some of the user data (users will need to provide an API key for an account on another site that I will use to pull data into the application).
At the moment, the main issue I'm having is figuring out what goes where. I've found plenty of resources that explain how to create an app using passport, but nothing shows how it would be incorporated into your website or where the files should be in relation to your website. I'm relatively new to Node.js, and while I've written a few small applications I have never hosted them anywhere.
Bonus question: I'm using MongoDB with passport and I was also planning to use it to store some JSON my application will be receiving from API calls. However, I wanted to use MySQL to store some data as well. More specifically, I'm planning to save the raw JSON then I'll create a relational database out of the data I need from the JSON and then keep the rest in MongoDB for easy access. Is this common/smart, or should I focus on keeping everything in my MongoDB? I'm relatively new to NoSQL.
Thanks in advance for any help.
I would reference this tutorial. I just recently used this to help myself with a new application. Also there is an example of the same thing but in SQL here. So not sure what you mean by " where the files should be in relation to your website". The information related to to authentication should go in your database.
To your "bonus question" you can use two databases. The key here is to ask yourself why and what are the true needs for data, and how is this data accessed and used. From ground up I would like one and stick with it. If at some point later you realize a certain type of data would be better in a different database then you can add it.
Side note: look into an IDE such as webstorm to help you out.
i would like to ask this for my second project of web apps, we were told to design and create javascript app that gets the information from a pre-defined database, and allows to add more data to it, modify it, erase any register, and search. That's pretty easy, but the thing is that our teacher asked that in the moment the browser makes the query to the server and this access the database, the server should return the response in form of a JSON.
I do not know how this is done, thats why i'm asking for a bit of assistance, since i always saw JSONs like libraries that can be add to a Javascript app.
You should pick a technology for creating a web accessible server api.
One common one in the .Net world is WEB API. You can also look into NodeJs (maybe using Express).
Json is really just a data format. It serves the same role as xml, but using a different notation.
Some frameworks like Asp.net Web API do content negotiation and returns data in the format requested in the http header (content type). Other solutions might require you to manually serialize the return data into JSON.
Link http://www.asp.net/web-api
I'm developing an app that displays a list of products that are in stock from various electronics websites. I have two methods in mind for accomplishing this task.
Create a C# application that parses the html from the various websites and updates a SQL table, use a php script to get data from the SQL
table, display it in JSON format, and parse the JSON with the iPhone
Use the iPhone to parse html from each website, eliminating the need for a server and SQL database
I don't know all the pros and cons for each method, but I do know that hosting a web server with a SQL database costs money. However, using the iPhone to parse html from over 40 websites could be very slow, and use a lot of data.
Which one is the best method for creating an app like this? Am I even on the right track?
Thanks,
Miles
I would prefer the web service approach in which I would parse it on my own server and then serve a JSON representation to iPhone.
Why?
Because when something changes on the html you won't need to wait for Apple to confirm your app change, and user will not need to update his app.
Your First Option is much prefered with little change. You Don't Need C# TO parse the HTML as PHP has all options you need. IT can save data in Database and similary can output in your desired format.
As per the Cost, using Iphone may cost you more with data usage(Operator service Charges) as compare to web server hosting with PHP & mysql. Further you can have already optimised facilities of HTML parsing in own server with tendency to meet all complexities served on WEB.
I am doing a Web project, where I am using HTML5 and Django to store database in sqlite3. The part being, since I had to use python, I installed mod_python for apache2. Now the thing is I went through many sites for head start to store the data and retrieve it. When I check in google related to how to use mod_python I did get certain information on how I can send a request to a .py file and then execute it. But with Django documentation providing a tutorial which is to develop a polling webApp, I am not finding it measurable to the part I am trying to achieve.
I want to send the request from an Html file where I have a search box and some radio buttons, to Django through Apache2 (mod_python) and then access my database in sqlite3. Please could anyone give me brief idea on whether I am doing it right? Also if yes, can u give me a hint as in how can I send the request from HTML file to the database from Django and mod_python being used? I am a bit confused with Django tutorial.
Please if anyone can just let me know what exactly I should do! Or if any link with a proper guidance would be of great help.
Thank you.
Well, for starters, if you can install and use mod_wsgi instead of mod_python, you should -- mod_python hasn't been supported for years. I'd also strongly recommend that you use anything other than SQLite for a production web application -- SQLite doesn't cope well with multiple simultaneous transactions.
If you're stuck using mod_python, then see the Django documentation for using mod_python.
Once you've got that working, the rest of your questions can be answered by the Django Tutorial, particularly part 4, which covers the use of HTML forms with Django views and templates:
http://docs.djangoproject.com/en/1.3/intro/tutorial01/
http://docs.djangoproject.com/en/1.3/intro/tutorial02/
http://docs.djangoproject.com/en/1.3/intro/tutorial03/
http://docs.djangoproject.com/en/1.3/intro/tutorial04/