How can i know the geographic origin from a web page or URL? - html

i'm building a web crawler and i'm trying to figure out where is a web page from. I mean, i can check the domain (for example, .com.ar ar from Argentina) but there are other sites, that have other domains (.com, .net) that are argentinean too, an example of these is www.taringa.net. Is an Argentinean site but with a .net domain.
So how can i do it?
Thanks.

Geo-location by IP. Do a reverse
look-up on the IP address, and you can
get a geographical location. These
services cost money, and will only
tell you physically where the server
is hosted.
Do a whois on the domain. It will
tell you the where the website is
registered.
But remember,
There is no meaning to "where is a web page from". The web has no geographic boundaries. I can run a Spanish language site out of San Jose California, and register the domain contacts in Canada. You will have no way of knowing my site is aimed at Chilean users.

You can use a whois query on the command line - or make a request to whois.arin.net and whois.xxxx.xxx depending on the result. If I map www.taringa.net to an IP, I get this:
www.taringa.net. 300 IN A 190.210.132.53
and running whois on that:
whois 190.210.132.53
gives me a ton of output:
inetnum: 190.210.132/24
status: reallocated
owner: WIROOS SRL
ownerid: AR-WISR1-LACNIC
responsible: ALBERTO NAKAYAMA
address: GRAL MIGUEL DE AZCUENAGA, 71, 4 A
address: C1029AAA - BUENOS AIRES -
country: AR
phone: +54 011 30973059 [3059]
This should generally work for any ip.

Related

confused by strange web address especially by "ai" term in start

While making online shopping,i came across a website for Live customer chat
https://ai.alimebot.daraz.pk/
"ai" in start of this web address means artificial intelligence?? If yes why and how?Infact on that link,i had chatted with a csr that was human being
ai.alimebot.daraz.pk is a host name. If you are the domain owner you can create any host names and sub domain names that you want, provided you use legal characters for names.
For example, they could have created you-are-not-expected-to-understand-this.ai.alimebot.daraz.pk

Typo3 mailserver issues (Mailing list hitobito CRM)

We from the the Swiss umbrella association for youth parliaments (DSJ) use TYPO3 as the backbone of our website. Next to TYPO3, we also use the CRM software hitobito, which allows us to create "Abos" with "mailing lists". However, this service is currently not working since Hitobito has recently changed its mail server. I have already changed the server addresses manually in our 365 Admin microsoft account and the changes have been verified by the Hitobito support.
This is where TYPO3 comes into play. The support staff from Hitobito suspects that the mail server configurations must also be changed in TYPO3. I, as a layman, have no clue where to make such changes, however. I was hoping you could help me out here. I believe the following information must be updated in the TYPO3 configuration:
*For the new mail server:
crm.dsj.ch IN MX 10 app.hitobito.ch.
For the outgoing mail server:
crm.dsj.ch 3600 IN TXT "v=spf1 a:mxout.appuio.ch -all*
The information you gave has nothing to do with TYPO3 but is part of the domain record. You should approach your domain registrar (seems to be https://www.visol.ch/ according to whois) with that.
The 1st one is to designate the mail server app.hitobito.ch for all incoming mail to recipients ...#crm.dsj.ch (so-called MX record).
And the 2nd one is to lower the spam level for outgoing mails from senders ...#crm.dsj.ch from the server mxout.appuio.ch (so-called SPF).
Is your webserver supposed to send mails, too? If so and you have problems with receiving these mails, I suggest to use the InstallTool's "test mail" function and send a mail to https://www.mail-tester.com/ - a great tool to identify spam-related problems.

Google Maps : Autocomplete - Giving results on typing 00000. How to avoid this?

When i type 000 - Google Auto Complete gives suggestions for Addresses ?
Why is that so ? What configuration I am missing here ? I cant attach images as I from Behind a Corporate Firewall but i will suggest the steps
Open maps.google.co.in
In Address type 000 - or till when address start appearing in Auto-Complete
The same behavior is in my Web using Maps plugin
Sometimes addresses can contain 000. For example, I can see the following suggestion '000 East Rd, Herscher, IL, United States' or '00040 Metropolitan City of Rome, Italy'.
In India I can see the '000000, Surat, Gujarat, India' with place ID ChIJY24F8TJO4DsRxYHx189lhoY after execution of
https://maps.googleapis.com/maps/api/place/autocomplete/json?input=000&components=country%3AIN&key=YOUR_API_KEY
Place details
https://maps.googleapis.com/maps/api/place/details/json?placeid=ChIJY24F8TJO4DsRxYHx189lhoY&key=YOUR_API_KEY
shows the following position
As you can see this is establishment and it looks like the data is not correct. You can try to filter types by address to exclude establishments
https://maps.googleapis.com/maps/api/place/autocomplete/json?input=000&types=address&components=country%3AIN&key=YOUR_API_KEY.
For data related issues, you can also use the "Report a problem" link on the bottom right corner of the maps.google.com.
https://support.google.com/maps/answer/3094088
Hope it helps!

UPS surepost API

I am trying to integrate UPS SurePost API in PHP to get the Rates for SurePost services.
I have no idea where to get the API from and how to integrate it.
Any help would be greatly appreciated.
Thanks
Jayesh Vaya's answer is correct, but here is further detail. There are 4 separate service codes for SurePost rates.
92: UPS SurePost Less than 1LB
93: UPS SurePost 1LB or greater
94: UPS SurePost BPM
95: UPS SurePost Media Mail
When shipping with service code 92, the unit of measure of the package weight must be in ounces instead of pounds, rounded to the tens decimal place.
I found that the ounces unit of measure code was "OZS".
Note that your account with UPS must be enabled to ship with SurePost. The documentation for the SurePost Web Services API is available upon request from your UPS account executive.
Here is the UPS documentation for using their API:
Rating Service API
I hope this helps anyone who wasn't sure how to start, like I was yesterday! :)
Service Code 93 Is for Surepost
<AccessLicenseNumber>xxxx</AccessLicenseNumber>
<UserId>xxxx</UserId>
<Password>xxxx</Password>
</AccessRequest>
<RatingServiceSelectionRequest>
<Request>
<TransactionReference>
<CustomerContext>Bare Bones Rate Request</CustomerContext>
<XpciVersion>1.0001</XpciVersion>
</TransactionReference>
<RequestAction>Rate</RequestAction>
<RequestOption>Rate</RequestOption>
</Request>
<PickupType>
<Code>02</Code>
</PickupType>
<Shipment>
<Shipper>
<Address>
<PostalCode>123456</PostalCode>
<CountryCode>US</CountryCode>
</Address>
<ShipperNumber>123456</ShipperNumber>
</Shipper>
<ShipTo>
<Address>
<PostalCode>123123</PostalCode>
<StateProvinceCode>AL</StateProvinceCode>
<CountryCode>US</CountryCode>
</Address>
</ShipTo>
<ShipFrom>
<Address>
<PostalCode>123123</PostalCode>
<StateProvinceCode>123</StateProvinceCode>
<CountryCode>US</CountryCode>
</Address>
</ShipFrom>
<Service>
<Code>93</Code>
</Service>
<Package>
<PackagingType>
<Code>02</Code>
</PackagingType>
<Description>Rate Shopping</Description>
<Dimensions><UnitOfMeasurement><Code>IN</Code></UnitOfMeasurement><Length>20</Length><Width>10</Width><Height>10</Height></Dimensions>
<PackageWeight><UnitOfMeasurement>
<Code>LBS</Code>
</UnitOfMeasurement>
<Weight>20.20</Weight>
</PackageWeight>
</Package>
<RateInformation><NegotiatedRatesIndicator/></RateInformation></Shipment></RatingServiceSelectionRequest>
Update 2021
It is interesting that SurePost is out of the list of service codes in the official docs.
So the only info about the service code of this method is this topic.
Thanks for that!
🙌
The mentioned 9x codes are working now.
And yes, you need to switch from the Shop request option to the Rate to fetch rates for a needed code with additional Service block (example for the JSON API:
"Service": {
"Code": "93",
"Description": ""
}
To ship packages via UPS SurePost service, you must first have a UPS Account along with a UPS Developer Account to gain access to their XML API.
https://www.ups.com/upsdeveloperkit
The example by #ayesh is the format of the xml payload you would send with your request for Rates, which with the correct SERVICE CODES, will return SurePost specific rates (if available to your ShipTo destination).
But ...
SurePost is not normally available to low volume UPS shippers and probably wouldn't be available to new shippers (not until enough volume is determined by UPS). SurePost must be added to an existing UPS account, which is done by contacting UPS and filling out additional applications.
Hope that helps!

How can I get the position (lat, lon) of a foursquare checkin posted on twitter?

I search with the twitter API all the public tweets with "4sq.com":
http://search.twitter.com/search.json?q=4sq.com
so I get all the check-in on Foursquare that actually shared on Twitter.
{"created_at":"Sun, 08 Jan 2012 20:05:27 +0000","from_user":"fenavergara","from_user_id":50876086,"from_user_id_str":"50876086","from_user_name":"Fernanda Vergara Ch","geo":null,"id":156104246239051776,"id_str":"156104246239051776","iso_language_code":"lt","metadata":{"result_type":"recent"},"profile_image_url":"http://a2.twimg.com/profile_images/1708744785/image_normal.jpg","profile_image_url_https":"https://si0.twimg.com/profile_images/1708744785/image_normal.jpg","source":"<a href="http://foursquare.com" rel="nofollow">foursquare</a>","text":"I'm at Roca Roja (Antofagasta) http://t.co/DIKdYGwD","to_user":null,"to_user_id":null,"to_user_id_str":null,"to_user_name":null},{"created_at":"Sun, 08 Jan 2012 20:05:27 +0000","from_user":"Fabioland76","from_user_id":357326176,"from_user_id_str":"357326176","from_user_name":"Fabio Landini","geo":null,"id":156104246104821760,"id_str":"156104246104821760","iso_language_code":"no","metadata":{"result_type":"recent"},"profile_image_url":"http://a3.twimg.com/profile_images/1506958638/profiloJPG_normal.jpg","profile_image_url_https":"https://si0.twimg.com/profile_images/1506958638/profiloJPG_normal.jpg","source":"<a href="http://foursquare.com" rel="nofollow">foursquare</a>","text":"I'm at Burger King (Gallerie Cantoni, Legnano) http://t.co/rU1jIJ4A","to_user":null,"to_user_id":null,"to_user_id_str":null,"to_user_name":null}
With these tweets, Foursquare provides a link (like this one: http://t.co/DIKdYGwD) where I can see the position of the check-in on a map (if I display the link on a webpage).
How can I get the latitude and longitude programmaticly directly from the list of tweets?
Idea: Do I need to fetch the link on a webpage and analyse it to get the latitude and longitude?
EDIT: I don't want to use Twitter geocode
There's actually a much nicer solution here if you would like to use the foursquare API. One that doesn't depend upon a user having enabled Twitter's geo-tagging.
You noticed already that the link in the status update redirects to a foursquare page. Rather than screen scraping, you may notice there are two key pieces of information in the url. The checkin id, and a parameter called s (for signature) that lets you access that checkin via foursquare's API.
https://foursquare.com/dens/checkin/4f0b3676e4b0e8ed3a590e41?s=MWHM3-0BsLXIJprfbALvOM606ZE&ref=tw
For instance, in the above url, the checkin id is 4f0b3676e4b0e8ed3a590e41 while the signature is MWHM3-0BsLXIJprfbALvOM606ZE
Using the /checkins endpoint, you can get the full checkin details (including lat/long of the venue). Dox here: https://developer.foursquare.com/docs/checkins/checkins
You already have what you want, look at the "geo" section in sample of search output you posted. It's null because it is not filled (twitter user forget to enable twitter location in settings and all locations from foursquare are ignored by twitter).
You can filter twits by location, look at the Twitter Dev documentation.
UPD:
If you want to get location even in case when it is not stored to twit, you have to perform request to foursquare short link and parse venue information (you can steal location also). Look at the source of the sample 4sq.com page (small part):
...
options['lat'] = 42.98758759825106;
options['lng'] = -71.50742411613464;
options['fuzzy'] = false;
options['venue'] = {"id":"v4f08df964fc62d44946f058a","venue":{"id":"4f08df964fc62d44946f058a","name":"ABC News \u2013 WMUR Debate","contact":{"twitter":"MittRomney"},"location":{"address":"100 St Anselm Drive","lat":42.98758759825106,"lng":-71.50742411613464,"city":"Manchester","state":"NH"},"categories":[{"id":"4bf58dd8d48988d1af941735","name":"College Auditorium","pluralName":"College Auditoriums","shortName":"Auditorium","icon":{"prefix":"https:\/\/foursquare.com\/img\/categories\/education\/default_","sizes":[32,44,64,88,256],"name":".png"},"primary":true}],"verified":false,"stats":{"checkinsCount":7,"usersCount":7,"tipCount":1}}};
...
Unfortunately, Foursquare team can change the format of the page or limit requests from IP Address without any notes. That's why this solution is ugly.
UPD2
See answer from #MikeLewis, I've missed that Foursquare API supports requesting information by checking id.