I am using MediaWiki to create a commonplace book. For the "author" pages I would like to include the infobox from that author's page on Wikipedia. Is there a routine, automated way to embed (transclude?) an infobox from Wikipedia on a page in my own MediaWiki installation?
Not as such. You can enable scary transcluding but that will only let you transclude full pages (or maybe full sections when used with
Labeled Section Transclusion). You can generate the infobox from Wikidata, but that's not always the same data you see on Wikipedia.
Related
I wish to determine whether a given Wikipedia page belongs to a certain Wikipedia Portal using the MediaWiki API. So far, I have been experimenting with the page properties of the API but I cannot seem to find a way to derive what Portal a given page belongs to.
As an example, on the Wikipedia page for Cake in the very bottom of the page, I can press Show on the section Cakes, and a bunch of links to different cake pages show up. There I can also see that all of these belong to the Food portal. It is that information that I would wish to extract from a given page using the MediaWiki API.
As far as I know, there is actually no formal definition of "belongings to a portal" in Wikipedia. Opposed to categories which are part of the MediaWiki software, portals are custom pages for Wikipedia that are aimed to make it easier to explore a topic.
Instead of a formal definition though, you can use an heuristic and determine the connection between the page and some portal based on one of them linking to the other. There are API endpoints for both:
(Note: 100 is the id of the 'Portal` namespace)
Which portal pages are linked from the page "Cake" or "Pizza"
https://en.wikipedia.org/w/api.php?action=query&format=json&prop=links&titles=Cake%7CPizza&plnamespace=100
Which portal pages link to the page "Cake" or "Pizza"
https://en.wikipedia.org/w/api.php?action=query&format=json&prop=linkshere&titles=Cake%7CPizza&lhnamespace=100
(though as you can see, many unrelated portals link to "Cake" and none link to "Pizza")
A combined query for both directions
https://en.wikipedia.org/w/api.php?action=query&format=json&prop=links%7Clinkshere&titles=Cake%7CPizza&plnamespace=100&lhnamespace=100
So trough some more investigation i found the answer:
I ended up using the Revisions property in the API. This allows me to to give a series of page titles that I want to investigate, and have the HTML of each page returned to me in json format. Then I can just search for lines containing Portal and figure out what portal (if any) the page belongs to.
If anyone are in a similar situation, here is an example query to the API:
https://en.wikipedia.org/w/api.php?action=query&prop=revisions&titles=Bread|Bubble_tea|Pizza&format=json&redirects&rvprop=content&rvslots=main
Mediawiki has the great {{Special:Recent pages}} template that you can transclude to just show a certain number of pages. However, I'd like a simple list of the latest pages created by users to display on the home page. Is there a way to do this easily? Perhaps something with the dynamic lists extension?
Have you looked at answers to this question ?
Embedding Recent Changes on Main Page on MediaWiki
You have several options :
The Dynamic Article List extension
The News extension
Use the Semantic Media Wiki extension and query your pages using the Modified Date property
I've created website with dynamic content, and I want google to know all my pages, so I've given a file "mysitemap.xml" via webmaster tools.
Basically, my links are like mysite.com/one-id/one-name , with one-id an id between 1 and 2000 (but will be greater with the time...).
I'm wondering if I need to create a page on my website (a kind of html sitemap), which will list all these links to help google bots to find my web pages, or is it enough for google to have the xml sitemap?
The problem is that the html sitemap will be very ugly and only a "for google" page, so I want to avoid this...
No, Google only requires a sitemap.
Sitemap is for Search Engines and Navigation is for humans.
Sitemap includes page's content type, update frequency, last modified, etc.
Navigation may include dropdown menus, hyperlinks, etc.
Is there a widely used standard way on how to index ajax loaded content (for search engines)?
For example, indexing HTML content that would dynamically be inserted into a page.
Thanks
You may want to consider using some sort of sitemap generator that aggregates all the content you normally load through AJAX.
Sitemaps are particularly beneficial
on websites where:
Some areas of the website are not available through the browsable
interface, or
Webmasters use rich Ajax, Silverlight, or Flash content that is
not normally processed by search
engines.
From Wikipedia - Sitemaps
Remember that:
Because most web crawlers do not
execute JavaScript code, publicly
indexable web applications should
provide an alternative means of
accessing the content that would
normally be retrieved with Ajax, to
allow search engines to index it.
From Wikipedia - AJAX Drawbacks
In addition you may be interested in checking out the following articles:
Official Google Webmaster Central Blog - A proposal for making AJAX crawlable
SoftwareDeveloper.com - How to: Get Google and AJAX to Play Nice
Crawling Ajax-driven Web 2.0 Applications
One way of doing this is using JS fallbacks for dialog boxes like thickbox: A link would point to the dialog box loading Ajax content, and the fallback href='...' would point to a search engine-readable representation of that content (i.e. the HTML snippet that the AJAX function would load, but surrounded by the necessary HTML body basics).
Example (I pulled rel='box' out of my arse, this is supposed to be the anchor for the box plugin, like rel=thickbox):
<a href='/encyclopedia/definition/mushroom.html' rel='box'>Definition of Mushroom</a>
Clicking on the link in a Ajax/JS enabled browser will open a nice dialog box with the article
Clicking on the link without JS (or as a search engine) will lead to a new page containing the article (which needs some server side intelligence to detect which channel the request came from).
That's all that comes to my mind in this direction. Ajax and search engines is a widely uncharted field otherwise.
Have Javascript fallbacks. Have a look at Amazon Diamond Search with and without Javascript enabled. Read up on http://www.seroundtable.com/archives/006889.html
I don't really know the answer, but it seems to me that ajax-loaded content won't help to improve se positions because search engine can't refer to ajax-loaded content. Another words search engine can't say: "Hey, go here and then click 3rd button from the top to see the content you're interested in.".
I think that good idea is to put this content to xml and put link to this xml at tag (like URL to RSS)...
What about using an alternative content for JS disabled clients (search engines)? I think there is no other way of letting the search engines index your AJAX site properly.
I think actually only Google really implements a specification to index AJAX content.
It's the Google AJAX crawling specification.
We have used that for our website, there is an example in our technical blog on how to do that with Django in a clean way.
What I mean by autolinking is the process by which wiki links inlined in page content are generated into either a hyperlink to the page (if it does exist) or a create link (if the page doesn't exist).
With the parser I am using, this is a two step process - first, the page content is parsed and all of the links to wiki pages from the source markup are extracted. Then, I feed an array of the existing pages back to the parser, before the final HTML markup is generated.
What is the best way to handle this process? It seems as if I need to keep a cached list of every single page on the site, rather than having to extract the index of page titles each time. Or is it better to check each link separately to see if it exists? This might result in a lot of database lookups if the list wasn't cached. Would this still be viable for a larger wiki site with thousands of pages?
In my own wiki I check all the links (without caching), but my wiki is only used by a few people internally. You should benchmark stuff like this.
In my own wiki system my caching system is pretty simple - when the page is updated it checks links to make sure they are valid and applies the correct formatting/location for those that aren't. The cached page is saved as a HTML page in my cache root.
Pages that are marked as 'not created' during the page update are inserted into the a table of the database that holds the page and then a csv of pages that link to it.
When someone creates that page it initiates a scan to look through each linking page and re-caches the linking page with the correct link and formatting.
If you weren't interested in highlighting non-created pages however you could just have a checker to see if the page is created when you attempt to access it - and if not redirect to the creation page. Then just link to pages as normal in other articles.
I tried to do this once and it was a nightmare! My solution was a nasty loop in a SQL procedure, and I don't recommend it.
One thing that gave me trouble was deciding what link to use on a multi-word phrase. Say you had some text saying "I am using Stack Overflow" and your wiki had 3 pages called "stack", "overflow" and "stack overflow"....which part of your phrase gets linked to where? It will happen!
My idea would be to query the titles like SELECT title FROM articles and simply check if each wikilink is in that array of strings. If it is you link to the page, if not, you link to the create page.
In a personal project I made with Sinatra (link text) after I run the content through Markdown, I do a gsub to replace wiki words and other things (like [[Here is my link]] and whatnot) with proper links, on each checking if the page exists and linking to create or view depending.
It's not the best, but I didn't build this app with caching/speed in mind. It's a low resource simple wiki.
If speed was more important, you could wrap the app in something to cache it. For example, sinatra can be wrapped with the Rack caching.
Based on my experience developing Juli, which is an offline personal wiki with autolink, generating static HTML approach may fix your issue.
As you think, it takes long time to generate autolinked Wiki page. However, in generating static HTML situation, regenerating autolinked Wiki page happens only when a wikipage is newly added or deleted (in other words, it doesn't happen when updating wikipage) and the 'regenerating' can be done in background so that usually I don't matter how it take long time. User will see only the generated static HTML.