For example, facebook. I have a list of different teams that shows overall data for that team, but the goal is the user will click on their team and send them to page that is detailed information about their team. My client wants the user's team name to be within the url so they can save the page as a favorite.
How I do this with out making a html file for every single team that gets made?
Im using Django for back end.
If you're using django then there is certainly no sense in making html for each team. You should make a template and populate this template with the data you're getting from your database (models.py) according to url (urls.py) and the appropriate view (views.py).
This is fairly basic django usage covered extensivly in the offical tutorials and the django documentation. Read it and use it, cause there is no shortcut. And last but not the least - enjoy cause such good tutorials and docs you won't see every day.
Related
Currently, I'm working with Beautiful Soup to extract information from websites. I've managed to scrape data from multiple pages from a certain apartment renting website with it - let's call it “Website A”. Now, I'd like to obtain data from another renting websites (“Website B”). I tried to follow a similar procedure as previously, but it failed because Website B requires a login.
I did manage to scrape the first page of apartments of Website B by means of Adelin's answer. His/her approach is based on the usage of Curl Trillworks (link). In principle, this approach could work for Website B as well. However, then one would need to manually repeat the procedure for the 800 or so pages on which the apartments are listed, and afterwards do the same for each of the 15 apartments per page.
This is too much work for me, so I try to automate the process. For instance, I tried adapting this to my situation, but I haven't succeeded so far. The dictionary I get is empty. I also tried making a new header for each page by putting a new referer each time in the original header. Then I'd put these referers in the the header dictionary. However, this failed - probably because websiteB recognized I was using the same cookie everytime I sent a request (the same one I used for the original apartment page for Website B).
So my question is:
Suppose one would have a list of pages of Website B that all have the same format (www.websiteB.com/PageNumber/ ). How would one
quickly/automatically obtain a header for each page by means of your
own login credentials for the website, with which one can create an appropriate response?
I could share the code I have so far, but I'm somewhat hesitant as this is a large commercial website and I suspect they aren't particularly happy with me sharing code that allows their website to be scraped and names the website itself as well.
I'm new to iOS development and I'm trying to make an app that tracks my student loans. I would like to have the app simply display a balance found on the loan's website and build on that behavior. I've got my app to open up safari to the website but I'd like more of a web-crawling behavior so it's done in the background. This web-crawler needs to be able to login to the website and then find a field with the loan balance value..
I've looked up several ways on how to do this but I can't get any of them to work with my novice level of experience with swift. If anyone has a recommendation and a direction to go with this, that would be great.
Thanks in advance!
I've written an app to get my uni's schedules, payment records, and grades by bypassing the login and gain access to API used by the website.
The step you could take are probably very similar to mine
Find out how the website authenticate its users (could be by cookie, session id's etc) You need to have some knowledge on web too. You can use the network tab on Chrome/Brave/Safari or any other browser to see what the website is doing when you click on 'login' for example. You could use Charles too, 30 days of trial should be enough.
After authentication succeed, track what API route the website use to get the datas to html. See what information you need to access the API route. Postman would be very handy to see the JSON response by the API.
To parse the response, you can use URLSession or networking library like Alamofire to get the job done.
If you want to see how I did it, here's the GitHub repo.
I am a naive user.
There is this website which is a really important source of information for my business.
To monitor the websites, I convert them to RSS feeds using page2rss service and then monitor feeds in IFTTT.
However, this particular site does not use static web pages and generates data response to API Calls:
Here is a sample API Call:
https://www.mpeproc.gov.in/ROOTAPP/GetTenderFreeView.jsp?Department=Urban%20Administration%20and%20Development%20Department&company=MPSEDC
Is there a way by which I could record the response from this call to an html page on my server? or is there any other way to monitor such dynamic pages.
There are solutions but not simple ones. The first page uses JavaScript to create a FORM which it then submits. You can simulate this with the command line tool curl; see https://superuser.com/questions/149329/what-is-the-curl-command-line-syntax-to-do-a-post-request
But take note that many sites don't like scraping; if they notice what you're doing, you may end up on a blacklist. So it's better to ask the site's owner for permission before you aim automated tools at their precious data.
I am trying to add the tab to a page I am admin of.
I use the url to do that -
http://www.facebook.com/dialog/pagetab?app_id=&next=.
Facebook shows a list of all the pages I am admin of. And that drop down has no specific sorting order.
Now my problem is - I have multiple pages with same page name. They ofcourse have different urls. I tried changing the name of pages, but due to high number of likes I can't change the names.
The only option I am left with is hit & Trial. And I have to do it for more than 30 apps.
So you understand my pain point.
Please advice any alternative.
Thanks
Pankaj
I would recommend writing down the page ids and making some sort of system for yourself to remember (perhaps only the last few digits) which page is which.
In any case, there is a way for you to add a tab application directly to a page without ever seeing that "Add Page Tab" dialog. You can do it all through the API. This means you'll need your pages access token so head on over to the Graph API Explorer, make sure you click the "get access token" button and mark the manage_pages permission.
You need to query /me/accounts to get a list of all the pages you administer.
You'll see a list with the page id, name, category... I hope you will be able to identify your page more easily here. Once you have, you'll need to get the access_token for that page. Keep a record of it - we'll need it in a few minutes. You'll also need the page id.
Modify the following URL to include the parameters we got previously -
https://graph.facebook.com/PAGE_ID/tabs?app_id=TAB_APP_ID&method=post&access_token=PAGE_ACCESS_TOKEN
Navigate to that URL and if all goes well, you'll get a simple true message indicating that the action was successful.
I am attempting to implement the following for my school's website:
School registration is currently a hard-copy form.
They'd like applicants to fill out a PDF form online.
They then print, sign and mail it to the school.
The school want to be able to store the data entered by the applicant in the PDF so that when the signed application arrives, they can simply approve the registration for that student without reentering all the students info.
The school system is run on MS Access (if it matters which version I can find this out).
Any ideas how I can implement an import from their form to MS Access?
Thanks.
I've used a website called www.doculicious.com that does pretty much this exact thing. You can upload a PDF and have it converted into a web form. Whoever fills in the web form can download the completed PDF and print it out for signing. All the data is stored in the system and you can get it out as XML or CSV for importing into another database manually, or use the API to do that part automatically.
It depends on how automatically you want it to be. A fully automated case would be
Scan in the printed form using TWAIN
Use a Form Recognition library to align the form back to the original and find the zones with the information you want
Use OCR to get the information out of the form (probably ICR, if handwritten)
Find the original record in Access
Update the record with the new data
This isn't simple -- there are products out there that could help, but they are not cheap, and ICR is not very accurate.
I would suggest that you just do #4 and #5 in Access and give them a screen where they process an incoming form -- someone keys in the new data. That form
Lets you specify the record you want to update (like with a name or ssn)
Shows you the record with edit boxes for you to fill in with the form data
Saves it back when you are done.
This is straight-forward in Access.
The school can use a regular HTML form on their website and ask parents to submit details online. Your server-side can save the submitted details to your database and generate a unique submission token/key/code back to the parent on the web page. When the parents come to the school and mention the key, print their details on paper and ask them to sign it. You file the paper and the data is already in your database.
If the intent is to import from an electronically submitted form, I have used Adobe LifeCycle Designer to accomplish this very thing. Basically you can create an XML output of the PDF form data using an email client of the end users choice and have the data from the XML file read from the inbox into the fields of your MS Access database. Of course this solution has some serious constraints on what applications are to be used, as well as what use cases are handled, for example this solution is in a controlled environment in which all users are required to email there requests, where possibly your requirements may need to handle mailed correspondence as well.
I would add to VSU's answer by suggesting the use of Google Forms for data entry.
You just need a free account, then use Docs to create a new Form.
It's extremely easy to setup, and the submitted results en-up in a Google spreadsheet that you can download as an regular Excel file (that makes it very easy to import into Access).
You have a few ways to share the form:
Email a link to the form
Embed the form into an email
Embed the form in your web site, which is probably the best solution for you.
Once the data is in your database, you can easily create a report and a simple lookup form to find the name of the applicant and print the form.
It shouldn't take more than one hour to get start and have something functional.
You can always improve later once you've experimented with this.
The advantage of using Google forms is that it's free, it works well, and you are fairly secure since you're not linking your database directly to the Internet (and don't risk SQL injection).