I would like to parse a webpage to get the content of a table on that page. However, the table is created by calling document.write() and several javascript functions. i.e. when I load the URL request of that page, the source that I get contains no html tags of that table. Is it possible to get the source of the final page (i.e. the source containing the html tags of the table I want)? And it's very curious that I can view the "final page source" in safari Browser's Developer Tools but not the source of the page.
this is the source of the page the source
You should use implement the NSConnectionDataDelegate protocol and use the NSConnection class. I believe that by the time -(void) connectionDidFinishLoading:(NSURLConnection *)connection is called, the page has finished loading completely.
-(void) requestPage
{
NSString *urlString = #"http://the.page.you.want.com";
NSURL *url = [NSURL URLWithString:urlString];
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:url cachePolicy:NSURLCacheStorageAllowed timeoutInterval:20.0f];
responseData = [[NSMutableData alloc] init];
connection = [[NSURLConnection connectionWithRequest:request delegate:self] retain];
delegate = target;
}
-(void) connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response
{
if ([response isKindOfClass:[NSHTTPURLResponse class]])
{
NSHTTPURLResponse *httpResponse = (NSHTTPURLResponse*) response;
//If you need the response, you can use it here
}
}
-(void) connection:(NSURLConnection *)connection didReceiveData:(NSData *)data
{
[responseData appendData:data];
}
-(void) connection:(NSURLConnection *)connection didFailWithError:(NSError *)error
{
[responseData release];
[connection release];
}
-(void) connectionDidFinishLoading:(NSURLConnection *)connection
{
if (connection == adCheckConnection)
{
NSString *responseString = [[NSString alloc] initWithData:responseData encoding:NSUTF8StringEncoding];
//You've got all the data now
//Do something with your response string
[responseString release];
}
[responseData release];
[connection release];
}
yes, just use view generated source in firefox with the web developer extension or firebug extension
Related
I'm a beginner in the area to work with database on iOS. However I could find a way to connect to a MySQL database, download and parse the json feed. Now in iOS 9, I can't use NSURLConnection anymore, that's why I have to replace it with NSURLSession. I saw many tutorials for example this here. So far, I was not able to replace it. Because I'm under time pressure, I can't waste more time to do this. Is here anyone who could help me to replace it?
My code looks exactly like this:
- (void)downloadItems
{
// Download the json file
NSURL *jsonFileUrl = [NSURL URLWithString:#"http://myhost.ch/test.php"];
// Create the request
NSURLRequest *urlRequest = [[NSURLRequest alloc] initWithURL:jsonFileUrl];
// Create the NSURLConnection
[NSURLConnection connectionWithRequest:urlRequest delegate:self];
}
#pragma mark NSURLConnectionDataProtocol Methods
- (void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response
{
// Initialize the data object
_downloadedData = [[NSMutableData alloc] init];
}
- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data
{
// Append the newly downloaded data
[_downloadedData appendData:data];
}
- (void)connectionDidFinishLoading:(NSURLConnection *)connection
{
// Create an array to store the locations
NSMutableArray *_locations = [[NSMutableArray alloc] init];
// Parse the JSON that came in
NSError *error;
NSArray *jsonArray = [NSJSONSerialization JSONObjectWithData:_downloadedData options:NSJSONReadingAllowFragments error:&error];
// Loop through Json objects, create question objects and add them to our questions array
for (int i = 0; i < jsonArray.count; i++)
{
NSDictionary *jsonElement = jsonArray[i];
// Create a new location object and set its props to JsonElement properties
Location *newLocation = [[Location alloc] init];
newLocation.idS = jsonElement[#"idStatistic"];
newLocation.temp = jsonElement[#"temp"];
newLocation.hum = jsonElement[#"hum"];
newLocation.date_time = jsonElement[#"date_time"];
// Add this question to the locations array
[_locations addObject:newLocation];
}
// Ready to notify delegate that data is ready and pass back items
if (self.delegate)
{
[self.delegate itemsDownloaded:_locations];
}
}
You can try this,
{
NSURL *url = [NSURL URLWithString:#"http://myhost.ch/test.php"];
NSMutableURLRequest *theRequest = [NSMutableURLRequest requestWithURL:url];
[theRequest setHTTPMethod:#"POST"];
NSURLSession *session = [NSURLSession sharedSession];
NSURLSessionDataTask *task = [session dataTaskWithRequest:theRequest
completionHandler:
^(NSData *data, NSURLResponse *response, NSError *error) {
responseDict = [NSJSONSerialization JSONObjectWithData:data options:kNilOptions error:&error];
NSLog(#"Result:%#", responseDict);
}];
[task resume];
}
I'm building an app for my Drupal based website. I currently have my iOS application pulling data from the MySQL database (e.g. Article titles and descriptions). Does anyone know what sort of code I would use to display an article's corresponding image (uploaded via a Drupal field) in an UIImageView?
Here's a quick snippet of the code I'm using to pull text from my Drupal site.
DoctorsViewController.h
#interface DoctorsViewController : UIViewController {
IBOutlet UITableView *DoctorsTableView;
NSArray *Doctors;
NSMutableData *data;
}
DoctorsViewController.m
- (void)viewDidLoad
{
[super viewDidLoad];
// Do any additional setup after loading the view from its nib.
[UIApplication sharedApplication].networkActivityIndicatorVisible = YES;
NSURL *url = [NSURL URLWithString:#"MY URL HERE"];
NSURLRequest *request = [NSURLRequest requestWithURL:url];
[[NSURLConnection alloc] initWithRequest:request delegate:self];
// Do any additional setup after loading the view, typically from a nib.
}
- (void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response
{
data = [[NSMutableData alloc] init];
}
- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)theData
{
[data appendData:theData];
}
- (void)connectionDidFinishLoading:(NSURLConnection *)connection
{
[UIApplication sharedApplication].networkActivityIndicatorVisible = NO;
Doctors = [NSJSONSerialization JSONObjectWithData:data options:nil error:nil];
[DoctorsTableView reloadData];
}
- (void)connection:(NSURLConnection *)connection didFailWithError:(NSError *)error
{
UIAlertView *errorView = [[UIAlertView alloc] initWithTitle:#"Error" message:#"The download could not complete - please make sure that you're connected to 3G or Wi-Fi." delegate:nil
cancelButtonTitle:#"Dismiss" otherButtonTitles:nil];
[errorView show];
[UIApplication sharedApplication].networkActivityIndicatorVisible = NO;
}
- (int)numberOfSectionsInTableView: (UITableView *)tableview
{
return 1;
}
- (int)tableView:(UITableView *)tableView numberOfRowsInSection:(NSInteger)section
{
return [Doctors count];
}
- (UITableViewCell *)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath
{
static NSString *DoctorsTableIdentifier = #"DoctorsCell";
DoctorsCell *cell = (DoctorsCell *)[tableView dequeueReusableCellWithIdentifier:DoctorsTableIdentifier];
if (cell == nil)
{
NSArray *nib = [[NSBundle mainBundle] loadNibNamed:#"DoctorsCell" owner:self options:nil];
cell = [nib objectAtIndex:0];
}
cell.firstnameLabel.text = [[Doctors objectAtIndex:indexPath.row] objectForKey:#"node_title"];
cell.descriptionLabel.text = [[Doctors objectAtIndex:indexPath.row] objectForKey:#"Opening Paragraph"];
return cell;
}
That isn't as straightforward as it sounds - Drupal stores its field data in a very specific way, and URIs to the images are addressed with a schema rather than having a relative or absolute path (e.g. public://path/to/image.jpg as opposed to /sites/default/files/path/to/image.jpg)
You can get the image URIs for a node straight from the DB like this (assuming your article type uses the standard field_image field):
SELECT fm.uri
FROM {file_managed} fm
INNER JOIN field_data_field_image fi ON fi.entity_type = 'node' AND fi.bundle = 'article' AND fi.deleted = 0 AND fi.field_image_fid = fm.fid
WHERE fi.entity_id = <nid>
Where <nid> is the node ID. From there you'll need to perform whatever logic you need to translate the paths to something useful for your iOS app. Drupal uses file_create_url() to perform the same task, so that's your best starting point for inspiration. Depending on your setup you may well get away with just doing a string replace on public:// to http://example.com/sites/default/files/.
Long term, you might want to look at adding the Services module and consuming the REST API in your app, instead of querying the DB directly. It'll take a lot of the strain of preparing Drupal's data out of your hands so you can just deal with what you need.
I am currently writing an iPhone application that will push a username and password to a website to retrieve the HTML source code of the page which loads. I include login information in NSString *post.
When I NSLog the _responseData instance variable in the connectionDidFinishLoading method, the console prints a long series of eight digit hexadecimal numbers which I assume are some type of address or encrypted HTML (3c21444f 43545950 45206874 6d6c2050 55424c49 4320222d 2f2f5733...).
What should I do to convert/decrypt the addresses into HTML code or otherwise retrieve the HTML code of the webpage which loads?
My ViewController conforms to the NSURLConnectionDelegate protocol and includes the following code:
#interface ViewController : UIViewController<NSURLConnectionDelegate>
{
NSMutableData *_responseData;
}
- (void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response {
// A response has been received, this is where we initialize the instance var you created
// so that we can append data to it in the didReceiveData method
// Furthermore, this method is called each time there is a redirect so reinitializing it
// also serves to clear it
_responseData = [[NSMutableData alloc] init];
NSLog(#"Received response");
}
- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data {
// Append the new data to the instance variable you declared
NSLog(#"Received data");
[_responseData appendData:data];
}
- (NSCachedURLResponse *)connection:(NSURLConnection *)connection
willCacheResponse:(NSCachedURLResponse*)cachedResponse {
// Return nil to indicate not necessary to store a cached response for this connection
return nil;
}
- (void)connectionDidFinishLoading:(NSURLConnection *)connection {
// The request is complete and data has been received
// You can parse the stuff in your instance variable now
//NSString *html=[NSString stringWithContentsOfURL:[NSURL URLWithString:#"https://grades.bsd405.org/Pinnacle/Gradebook/InternetViewer/GradeSummary.aspx?&EnrollmentId=770595&TermId=127463&ReportType=0&StudentId=114040"] encoding:NSASCIIStringEncoding error:nil];
NSLog(#"Finished loading");
//PRINT HTML:
NSLog(#"Data: %#",_responseData);
}
- (void)connection:(NSURLConnection *)connection didFailWithError:(NSError *)error {
// The request has failed for some reason!
// Check the error var
NSLog(#"Failed to load");
}
- (IBAction)goPressed:(UIButton *)sender {
NSString *post = #"__LASTFOCUS&__EVENTTARGET&__EVENTARGUMENT&__VIEWSTATE=/wEPDwUJNTkxNzI3MDIzD2QWAmYPZBYCA
gMPZBYGAgEPZBYCAgkPZBYCAgEPZBYIAgMPFgIeB1Zpc2libGVoZAIFDxYCHwBoZAIHDxYCHwBoZAIJDxYCHgVzdH
lsZQUjdmVydGljYWwtYWxpZ246bWlkZGxlO2Rpc3BsYXk6bm9uZTtkAgMPDxYCHwBoZGQCBQ9kFghmD2QWAgINDxY
CHgVjbGFzcwUQc2luZ2xlU2Nob29sTGlzdBYCAgEPZBYCAgEPEGQPFgFmFgEQBQ5EZWZhdWx0IERvbWFpbgUIUGlu
bmFjbGVnZGQCAg9kFgICEw9kFgICAQ9kFgICAQ8QZGQWAGQCBw8PFgIeBFRleHQFIFBpbm5hY2xlIEdyYWRlIDIwM
TIgV2ludGVyIEJyZWFrZGQCCA8PFgIfAwU3Q29weXJpZ2h0IChjKSAyMDEzIEdsb2JhbFNjaG9sYXIuICBBbGwgcm
lnaHRzIHJlc2VydmVkLmRkZP/l6irI9peZfyqpKjk3fwLuEbos&__EVENTVALIDATION=/wEWBgKjnbqUCQLnksmg
AQKTpbWbDgLB5+KIBAL4xb20BAK20ZqiCel6sQLBsF1W3XHOxpgq+tJj+Rx2&ctl00$ContentPlaceHolder$Use
rname=TESTUSERNAME&ctl00$ContentPlaceHolder$Password=TESTPASSWORD&ctl00$ContentPlaceHolde
r$lstDomains=Pinnacle&ctl00$ContentPlaceHolder$LogonButton=Sign
in&PageUniqueId=2dacba26-
bb0d-412f-b06a-e02caf039c4b";
NSData *postData = [post dataUsingEncoding:NSASCIIStringEncoding allowLossyConversion:YES];
NSString *postLength = [NSString stringWithFormat:#"%d", [postData length]];
NSMutableURLRequest *request = [[NSMutableURLRequest alloc] init];
[request setURL:[NSURL URLWithString:#"https://grades.bsd405.org/Pinnacle/Gradebook/Logon.aspx?ReturnUrl=%2fPinnacle%2fGradebook%2fDefault.aspx"]];
[request setHTTPMethod:#"POST"];
[request setValue:postLength forHTTPHeaderField:#"Content-Length"];
[request setValue:#"application/x-www-form-urlencoded" forHTTPHeaderField:#"Content-Type"];
[request setHTTPBody:postData];
[NSURLConnection connectionWithRequest:request delegate:self];
}
The NSLog of your NSData displays the object's description, which a hexadecimal string of the underlying binary data:
3c21444f 43545950 45206874 6d6c2050 55424c49 4320222d 2f2f5733 ...
Translates to a string of
<!DOCTYPE html PUBLIC "-//W3 ...
To get the NSString from the NSData, you do:
NSString *string = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
So i work with UICollectionView with my JSON. I write this code for parse, but i am not understand for why it's not work. My JSON have ([jsontest2] after key "imageMain").
So i paste my code, please help me:
#property (nonatomic,strong) NSMutableArray *patternImagesArray;
#end
#implementation ViewController
#synthesize patternImagesArray = _patternImagesArray;
NSURLConnection *connection;
NSMutableData *webdata;
Try to get data from JSON:
-(void) viewDidLoad{
_patternImagesArray = [[NSMutableArray alloc] init];
NSURL *url = [NSURL URLWithString:#"site"];
NSURLRequest *request = [NSURLRequest requestWithURL:url];
connection = [NSURLConnection connectionWithRequest:request delegate:self];
if (connection) {
webdata = [[NSMutableData alloc]init];
}
}
-(void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response
{
[webdata setLength:0];
}
-(void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data
{
[webdata appendData:data];
}
-(void)connection:(NSURLConnection *)connection didFailWithError:(NSError *)error
{
}
JSON parse:
-(void)connectionDidFinishLoading:(NSURLConnection *)connection
{
NSDictionary *allDataDictionary = [NSJSONSerialization JSONObjectWithData:webdata options:0 error:nil];
NSArray *tmp =[allDataDictionary objectForKey:#"jsontest2"];
if (tmp.count>0){
for (NSDictionary *diction in tmp) {
[self.patternImagesArray addObject:diction];
}
NSLog(#"%#", self.patternImagesArray);
}
[self.collectionView reloadData];
}
Some customize things:
-(UICollectionViewCell *)collectionView:(UICollectionView *)collectionView cellForItemAtIndexPath:(NSIndexPath *)indexPath{
PatternView *cell = [collectionView dequeueReusableCellWithReuseIdentifier:#"PatternCell" forIndexPath:indexPath];
NSString *myPatternString = [[self.patternImagesArray objectAtIndex:indexPath.row] valueForKey:#"thumb"];
NSLog(#"myPatternString %#",myPatternString);
NSData *data = [NSData dataWithContentsOfURL:[NSURL URLWithString:myPatternString]];
cell.patternImageView.image = [UIImage imageWithData:data];
return cell;
}
-(NSInteger)numberOfSectionsInCollectionView:(UICollectionView *)collectionView{
return 1;
}
-(NSInteger)collectionView:(UICollectionView *)collectionView numberOfItemsInSection:(NSInteger)section{
return [self.patternImagesArray count];
}
-(CGSize)collectionView:(UICollectionView *)collectionView layout:(UICollectionViewLayout *)collectionViewLayout sizeForItemAtIndexPath:(NSIndexPath *)indexPath{
return CGSizeMake(100.0, 100.0);
}
You are abusing the variable "array" for two totally different purposes.
The underlying reason is that you haven't bothered to find reasonable names for variables. "array" is actually a global variable. It's not only accessible from any code within your file, but from any code anywhere in your program. Look at this bit of code:
NSDictionary *allDataDictionary =[NSJSONSerialization JSONObjectWithData:_webdata
options:kNilOptions error:nil];
array =[allDataDictionary objectForKey:#"jsontest2"];
for (array in allDataDictionary) {
}
You first parse the JSON data to allDataDictionary (you don't bother checking for errors). You access one key "jsontest2" from the dictionary and store it into the global variable "array". Whatever was in "array" before is now gone.
Then you have a for-loop, using the same global variable as the loop variable! That's total nonsense. But you also need to realise that this will iterate over the keys in your dictionary, so array is now an NSString and not an array anymore.
After that your code gets worse...
Go through your code line by line. Every line, think about what it is doing. There are many severe problems there.
I spent a lot of time for research on POST Requests using Cocoa.
I found some code that looked good. I changed it like it fits my needs. But the code didn't work and I can't find the bug because I'm pretty new to cocoa.
Here is my code.
- (IBAction)sendForm:(id)sender
{
NSLog(#"web request started");
NSString *post = [NSString stringWithFormat:#"firstName=%#&lastName=%#&eMail=%#&message=%#", firstName.stringValue, lastName.stringValue, eMail.stringValue, message.stringValue];
NSData *postData = [post dataUsingEncoding:NSASCIIStringEncoding allowLossyConversion:YES];
NSString *postLength = [NSString stringWithFormat:#"%d", [postData length]];
NSMutableURLRequest *request = [[[NSMutableURLRequest alloc] init] autorelease];
[request setURL:[NSURL URLWithString:#"myDomain/form.php"]];
[request setHTTPMethod:#"POST"];
[request setValue:postLength forHTTPHeaderField:#"Content-Length"];
[request setValue:#"application/x-www-form-urlencoded" forHTTPHeaderField:#"Content-Type"];
[request setHTTPBody:postData];
NSURLConnection *theConnection = [[NSURLConnection alloc] initWithRequest:request delegate:self];
if(theConnection)
{
webData = [[NSMutableData data] retain];
NSLog(#"connection initiated");
}
}
- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data
{
[webData appendData:data];
NSLog(#"connection received data");
}
- (void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response
{
NSLog(#"connection received response");
NSHTTPURLResponse *ne = (NSHTTPURLResponse *)response;
if([ne statusCode] == 200)
{
NSLog(#"connection state is 200 - all okay");
NSString *html = [[NSString alloc] initWithBytes: [webData mutableBytes] length:[webData length] encoding:NSUTF8StringEncoding];
[[webView mainFrame] loadHTMLString:html baseURL:[NSURL URLWithString:#"myDomain"]];
}
}
But the only two NSLog messages I receive are "web request started" and "connection initiated".
So I think - (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data and - (void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)responseare not called.
Can anybody help me with this issue?
Thanks for help, Julian
You almost did it correctly!
I can see the post is a year old but in the hope that it may help others and for my own reference, I am posting this here.
Apple's documentation for connection:didReceiveResponse: says :-
Sent when the connection has received sufficient data to construct the
URL response for its request.
As with most Apple documentation this too is confusing and maybe misleading. Usually when we have "sufficient data to construct the URL response" means we have the complete response which further means that we have response body too; which is universally considered part of the response.
However in this case the NSHTTPURLResponse object does not have a body. So my guess is, what Apple means is - connection:didReceiveResponse: is invoked when we have received the response header which has "sufficient" data to let us known everything about the response, except the actual data (body).
In fact many times you will notice that connection:didReceiveData: gets called after connection:didReceiveResponse: is invoked.
So, we can modify your code as below :-
- (void)applicationDidFinishLaunching:(NSNotification *)aNotification {
NSLog(#"web request started");
NSString *post = [NSString stringWithFormat:#"firstName=%#&lastName=%#&eMail=%#&message=%#", firstName.stringValue, lastName.stringValue, eMail.stringValue, message.stringValue];
NSData *postData = [post dataUsingEncoding:NSUTF8StringEncoding];
NSString *postLength = [NSString stringWithFormat:#"%ld", (unsigned long)[postData length]];
NSLog(#"Post data: %#", post);
NSMutableURLRequest *request = [NSMutableURLRequest new];
[request setURL:[NSURL URLWithString:#"https://example.com/form.php"]];
[request setHTTPMethod:#"POST"];
[request setValue:postLength forHTTPHeaderField:#"Content-Length"];
[request setValue:#"application/x-www-form-urlencoded" forHTTPHeaderField:#"Content-Type"];
[request setHTTPBody:postData];
NSURLConnection *theConnection = [[NSURLConnection alloc] initWithRequest:request delegate:self];
if(theConnection) {
webData = [NSMutableData data];
NSLog(#"connection initiated");
}
}
- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data {
[webData appendData:data];
NSLog(#"connection received data");
}
- (void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response {
NSLog(#"connection received response");
NSHTTPURLResponse *ne = (NSHTTPURLResponse *)response;
if([ne statusCode] == 200) {
NSLog(#"connection state is 200 - all okay");
} else {
NSLog(#"connection state is NOT 200");
}
}
-(void) connection:(NSURLConnection *)connection didFailWithError:(NSError *)error {
NSLog(#"Conn Err: %#", [error localizedDescription]);
}
- (void)connectionDidFinishLoading:(NSURLConnection *)connection {
NSLog(#"Conn finished loading");
NSString *html = [[NSString alloc] initWithBytes: [webData mutableBytes] length:[webData length] encoding:NSUTF8StringEncoding];
NSLog(#"OUTPUT:: %#", html);
}
Have you tried to implement the error delegate method?
- (void)connection:(NSURLConnection *)connection didFailWithError:(NSError *)error {
// do something with error
}
It can happen that you don't have an internet connection, or something else is wrong before NSURLConnection is able to establish a connection. In this case you will not receive a http response or any other data. The error will provide more information.
[edit]
I just noticed the URL in the example code doesn't contain a host name. If this is the exact code you are using make sure you send the request to a correct URL.
// Replace
[NSURL URLWithString:#"myDomain/form.php"]
// with a correct url like
[NSURL URLWithString:#"http://mydomain.com/form.php"];
An error delegate method will actually provide you with an error that complains about a wrong URL.