Is there a way to convert Trac Wiki pages to HTML? - html

I see the suggestion of using Mylyn WikiText to convert wiki pages to html from this question except I'm not sure if its what I'm looking for from reading the front page of the site alone. I'll look into it further. Though I would prefer it being a Trac plug-in so I could initiate the conversion from within the wiki options but all the plugins at Trac-Hacks export single pages only whereas I want to dump all formatted pages in one go.
So is there an existing Trac plug-in or stand-alone application that'll meet my requirements? If not where would you point me to start looking at implementing that functionality myself?

You may find some useful information in the comments for this ticket on trac-hacks. One user reports using the wget utility to create a mirror copy of the wiki as if it was a normal website. Another user reports using the XmlRpc plugin to extract HTML versions of any given wiki page, but this method would probably require you to create a script to interface with the plugin. The poster didn't provide any example code, unfortunately, but the XmlRpc Plugin page includes a decent amount of documentation and samples to get you started.
If you have access to a command line on the server hosting Trac, you can use the trac-admin command like:
trac-admin /path/to/trac wiki export <wiki page name>
to retrieve a plain-text version of the specified wiki page. You would then have to parse the wiki syntax to HTML, but there are tools available to do that.

For our purposes, we wanted to export each of the wiki pages individually without the header/footer and other instance-specific content. For this purpose, the XML-RPC interface was a good fit. Here's the Python 3.6+ script I created for exporting the whole of the wiki into HTML files in the current directory. Note that this technique doesn't rewrite any hyperlinks, so they will resolve absolutely to the site.
import os
import xmlrpc.client
import getpass
import urllib.parse
def add_auth(url):
host = urllib.parse.urlparse(url).netloc
realm = os.environ.get('TRAC_REALM', host)
username = getpass.getuser()
try:
import keyring
password = keyring.get_password(realm, username)
except Exception:
password = getpass.getpass(f"password for {username}#{realm}: ")
if password:
url = url.replace('://', f'://{username}:{password}#')
return url
def main():
trac_url = add_auth(os.environ['TRAC_URL'])
rpc_url = urllib.parse.urljoin(trac_url, 'login/xmlrpc')
trac = xmlrpc.client.ServerProxy(rpc_url)
for page in trac.wiki.getAllPages():
filename = f'{page}.html'.lstrip('/')
dir = os.path.dirname(filename)
dir and os.makedirs(dir, exist_ok=True)
with open(filename, 'w') as f:
doc = trac.wiki.getPageHTML(page)
f.write(doc)
__name__ == '__main__' and main()
This script requires only Python 3.6, so download and save to a export-wiki.py file, then set the TRAC_URL environment variable and invoke the script. For example on Unix:
$ TRAC_URL=http://mytrac.mydomain.com python3.6 export-wiki.py
It will prompt for a password. If no password is required, just hit enter to bypass. If a different username is needed, also set the USER environment variable. Keyring support is also available but can be disregarded.

Related

Docstrings are not generated on Read the Docs with Sphinx autodoc and napoleon extensions

I am using the Sphinx autodoc and napoleon extensions to generate the documentation for my project (Qtools). This works well on my local machines. I am using Sphinx 3.1.2 (or higher). However, when I build the documentation on Read the Docs (RTD), only text added directly to the reStructuredText files that form the source of the documentation is processed. The docstrings that are supposed to be pulled in by autodoc do not appear in the HTML documentation generated by RTD. So for example in docs\source\section2_rsdoc.rst I have:
Response spectra
================
The response spectrum class
---------------------------
.. autoclass:: qtools.ResponseSpectrum
:members:
Response spectrum creation
--------------------------
.. autofunction:: qtools.calcrs
.. autofunction:: qtools.calcrs_cmp
.. autofunction:: qtools.loadrs
See also :func:`qtools.convert2rs` (converts a power spectrum into a response spectrum).
This results in:
Response spectra
The response spectrum class
Response spectrum creation
See also qtools.convert2rs (converts a power spectrum into a response spectrum).
In other words, all directives are apparently ignored, and hyperlinks to other functions are not added. I have examined several basic guidance documents such as this one, but I cannot figure out what I am doing wrong. RTD builds the documentation without any errors or warnings. In RTD advanced settings I have:
Documentation type: Sphinx HTML
Requirements file: requirements.txt
Python interpreter: CPython 3.x
Install Project: no
Use system packages: no
Python configuration file: blank
Enable PDF build: no
Enable EPUB build: no
I haven't touched any other settings.
In conf.py I have tried the following variations of line 15: sys.path.insert(0, os.path.abspath('.')), sys.path.insert(0, os.path.abspath('../..')) and the current sys.path.insert(0, os.path.abspath('../../..')). None of those made any difference.
I would be grateful for any help!
RTD builds the documentation without any errors or warnings
This is slightly incorrect. As you can see in the build logs, autodoc is emitting numerous warnings like this one:
WARNING: autodoc: failed to import class 'ResponseSpectrum' from module 'qtools'; the following exception was raised:
No module named 'qtools'
This has happened for all your variations of sys.path.insert, as you can see in some past builds.
Trying to make it work this way is tricky, since Read the Docs does some magic to guess the directory where your documentation is located, and also the working directory changes between commands.
Instead, there are two options:
Locate where the conf.py is located (see How do you properly determine the current script directory?) and work out a relative package from there.
Invest some time into making your code installable using up-to-date Python packaging standards, for example putting all your sources inside a qtools directory, and creating an appropriate pyproject.toml file using flit.

Embed Jupyter HTML output in a web page

I want to embed the HTML output of Jupyter, in my own web page. The reason for this is primarily, so that I can use Jupyter from my own webapp - and also access my research notebooks from anywhere in the world - via the internet.
A typical use case scenario would be that I click on a button on my page, and an iframe will be inserted in my page; Jupyter will then be launched at the backend (if not already running), and the output of Jupyter will be 'piped' to the iframe - so that I can use Jupyter from within my page.
The naive solution it appeared, was to use <iframe>, but there were two problems:
The iframe cross domain policy problem
Jupyter generated a one time authentication token when first launched
Is there anyway I can overcome these issues, so I can embed the output of Jupyter in my own web page?
you need to check nbconvert - https://github.com/jupyter/nbconvert
there you have 2 options.
to use command line to run notebook and then to let some web server
to server .html
to use python, and nbconvert library
here is short code :
if you want to show already generated:
from nbconvert.preprocessors import ExecutePreprocessor
import nbformat
from nbconvert import HTMLExporter
from nbconvert.preprocessors.execute import CellExecutionError
src_notebook = nbformat.reads(ff.read(), as_version=4) #where ff is file opened with some open("path to notebook file")
html_exporter = HTMLExporter()
html_exporter.template_file = 'basic' #basic will skip generating body and html tags.... use "all" to gen all..
(body, resources) = html_exporter.from_notebook_node(src_notebook)
print(body) #body have html output
if you want also to run notebook, then :
from nbconvert.preprocessors import ExecutePreprocessor
import nbformat
from nbconvert import HTMLExporter
from nbconvert.preprocessors.execute import CellExecutionError
src_notebook = nbformat.reads(ff.read(), as_version=4) #where ff is file opened with some open("path to notebook file")
ep = ExecutePreprocessor(timeout=50, kernel_name='python3')
ep.preprocess(src_notebook, {})
html_exporter = HTMLExporter()
html_exporter.template_file = 'basic' #basic will skip generating body and html tags.... use "all" to gen all..
(body, resources) = html_exporter.from_notebook_node(src_notebook)
print(body) #body have html output
You can directly do that using the html_embed pre-processor:
$ jupyter nbconvert --to html_embed Annex.ipynb
[NbConvertApp] Converting notebook Annex.ipynb to html_embed
/usr/local/lib/python3.6/site-packages/nbconvert/filters/datatypefilter.py:41: UserWarning: Your element with mimetype(s) dict_keys(['image/pdf']) is not able to be represented.
mimetypes=output.keys())
[NbConvertApp] Writing 2624499 bytes to Annex.html
Strangely, I could not find a direct reference in the manual from nbconvert.
You can use ipython nbconvert - -to html notebook.ipynb to obtain the html code for the same.
Here is a guide on how to do it Blogging with the IPython notebook - see here
If your website is writing in python the use python embed docs
Also this Tutorial - see here
or use kyso.io
Here is how to embed Jupyter using Kyso platform - see here
(disclaimer - I’m a founder of kyso)

how to get source code for read the docs html template?

how to get source code of html template of read the docs ?
I get sphinx package from git hub but i don't know python or how to proceed with that package
I simply want HTML and css files of that template then i will modify according to my requirement
If you want to use the ReadTheDocs theme locally with Sphinx you can clone/fork the code from the Github repository below.
Source: https://github.com/snide/sphinx_rtd_theme
If you're also building your Docs using readthedocs.org you'll need to enter the following to your conf.py to avoid issues with the RTD build process:
# on_rtd is whether we are on readthedocs.org, this line of code
grabbed from docs.readthedocs.org
on_rtd = os.environ.get('READTHEDOCS', None) == 'True'
if not on_rtd: # only import and set the theme if we're building docs locally
import sphinx_rtd_theme
html_theme = 'sphinx_rtd_theme'
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
Otherwise, readthedocs.org uses their theme by default, so no need to specify it.

Run native binary CGI on lighttpd

I'm trying to set up lighttpd to run binary CGI app (not PHP script or smth, but a binary file, compiled from C++ source). I actually have
server.modules = (
...
"mod_cgi"
...
)
uncommented, have myApp.exe in htdocs/app, and also
cgi.assign = ( "myApp.exe" => "myApp.exe" )
Then, to make all the stuff work by accessing, say, http://localhost:8080/app/myApp.exe?p=a&..., I had to put an empty myApp.exe in lighttpd root folder (where the server's exe is). It's actually strange and sucks, and also not all CGIs can work that way. Applying these actions to another CGI app (that works perfectly on properly tuned Apache) gave no success.
What am I doing wrong?
The docs: http://redmine.lighttpd.net/wiki/1/Docs:ModCGI
I've made a test with a tcl script as cgi and this was my working config:
cgi.assign = ( "" => "/usr/bin/tclsh" )
index-file.names = ("lighttd_test.tcl")
The cgi.assign allows you to specify file extensions to be handled by specific applications. This example means: Any filetype will be opened through /usr/bin/tclsh. Since my index-file is a tcl script, I get the content which I put through the script's STDOUT.
In case you want to run a binary executable this is the place to specify it.
Maybe this link provides some more info about binary cgi for you: http://redmine.lighttpd.net/issues/1256

Get the application's path

I've recently searched how I could get the application's directory in Java. I've finally found the answer but I've needed surprisingly long because searching for such a generic term isn't easy. I think it would be a good idea to compile a list of how to achieve this in multiple languages.
Feel free to up/downvote if you (don't) like the idea and please contribute if you like it.
Clarification:
There's a fine distinction between the directory that contains the executable file and the current working directory (given by pwd under Unix). I was originally interested in the former but feel free to post methods for determining the latter as well (clarifying which one you mean).
In Java the calls
System.getProperty("user.dir")
and
new java.io.File(".").getAbsolutePath();
return the current working directory.
The call to
getClass().getProtectionDomain().getCodeSource().getLocation().getPath();
returns the path to the JAR file containing the current class, or the CLASSPATH element (path) that yielded the current class if you're running directly from the filesystem.
Example:
Your application is located at
C:\MyJar.jar
Open the shell (cmd.exe) and cd to C:\test\subdirectory.
Start the application using the command java -jar C:\MyJar.jar.
The first two calls return 'C:\test\subdirectory'; the third call returns 'C:\MyJar.jar'.
When running from a filesystem rather than a JAR file, the result will be the path to the root of the generated class files, for instance
c:\eclipse\workspaces\YourProject\bin\
The path does not include the package directories for the generated class files.
A complete example to get the application directory without .jar file name, or the corresponding path to the class files if running directly from the filesystem (e.g. when debugging):
String applicationDir = getClass().getProtectionDomain().getCodeSource().getLocation().getPath();
if (applicationDir.endsWith(".jar"))
{
applicationDir = new File(applicationDir).getParent();
}
// else we already have the correct answer
In .NET (C#, VB, …), you can query the current Assembly instance for its Location. However, this has the executable's file name appended. The following code sanitizes the path (using System.IO and using System.Reflection):
Directory.GetParent(Assembly.GetExecutingAssembly().Location)
Alternatively, you can use the information provided by AppDomain to search for referenced assemblies:
System.AppDomain.CurrentDomain.BaseDirectory
VB allows another shortcut via the My namespace:
My.Application.Info.DirectoryPath
In Windows, use the WinAPI function GetModuleFileName(). Pass in NULL for the module handle to get the path for the current module.
Python
path = os.path.dirname(__file__)
That gets the path of the current module.
Objective-C Cocoa (Mac OS X, I don't know for iPhone specificities):
NSString * applicationPath = [[NSBundle mainBundle] bundlePath];
In Java, there are two ways to find the application's path. One is to employ System.getProperty:
System.getProperty("user.dir");
Another possibility is the use of java.io.File:
new java.io.File("").getAbsolutePath();
Yet another possibilty uses reflection:
getClass().getProtectionDomain().getCodeSource().getLocation().getPath();
In VB6, you can get the application path using the App.Path property.
Note that this will not have a trailing \ EXCEPT when the application is in the root of the drive.
In the IDE:
?App.Path
C:\Program Files\Microsoft Visual Studio\VB98
In .Net you can use
System.IO.Directory.GetCurrentDirectory
to get the current working directory of the application, and in VB.NET specifically you can use
My.Application.Info.DirectoryPath
to get the directory of the exe.
Delphi
In Windows applications:
Unit Forms;
path := ExtractFilePath(Application.ExeName);
In console applications:
Independent of language, the first command line parameter is the fully qualified executable name:
Unit System;
path := ExtractFilePath(ParamStr(0));
Libc
In *nix type environment (also Cygwin in Windows):
#include <unistd.h>
char *getcwd(char *buf, size_t size);
char *getwd(char *buf); //deprecated
char *get_current_dir_name(void);
See man page
Unix
In unix one can find the path to the executable that was started using the environment variables. It is not necessarily an absolute path, so you would need to combine the current working directory (in the shell: pwd) and/or PATH variable with the value of the 0'th element of the environment.
The value is limited in unix though, as the executable can for example be called through a symbolic link, and only the initial link is used for the environment variable. In general applications on unix are not very robust if they use this for any interesting thing (such as loading resources). On unix, it is common to use hard-coded locations for things, for example a configuration file in /etc where the resource locations are specified.
In bash, the 'pwd' command returns the current working directory.
In PHP :
<?php
echo __DIR__; //same as dirname(__FILE__). will return the directory of the running script
echo $_SERVER["DOCUMENT_ROOT"]; // will return the document root directory under which the current script is executing, as defined in the server's configuration file.
echo getcwd(); //will return the current working directory (it may differ from the current script location).
?>
in Android its
getApplicationInfo().dataDir;
to get SD card, I use
Environment.getExternalStorageDirectory();
Environment.getExternalStoragePublicDirectory(String type);
where the latter is used to store a specific type of file (Audio / Movies etc). You have constants for these strings in Environment class.
Basically, for anything to with app use ApplicationInfo class and for anything to do with data in SD card / External Directory using Environment class.
Docs :
ApplicationInfo ,
Environment
In Tcl
Path of current script:
set path [info script]
Tcl shell path:
set path [info nameofexecutable]
If you need the directory of any of these, do:
set dir [file dirname $path]
Get current (working) directory:
set dir [pwd]
Java:
On all systems (Windows, Linux, Mac OS X) works for me only this:
public static File getApplicationDir()
{
URL url = ClassLoader.getSystemClassLoader().getResource(".");
File applicationDir = null;
try {
applicationDir = new File(url.toURI());
} catch(URISyntaxException e) {
applicationDir = new File(url.getPath());
}
return applicationDir;
}
in Ruby, the following snippet returns the path of the current source file:
path = File.dirname(__FILE__)
In CFML there are two functions for accessing the path of a script:
getBaseTemplatePath()
getCurrentTemplatePath()
Calling getBaseTemplatePath returns the path of the 'base' script - i.e. the one that was requested by the web server.
Calling getCurrentTemplatePath returns the path of the current script - i.e. the one that is currently executing.
Both paths are absolute and contain the full directory+filename of the script.
To determine just the directory, use the function getDirectoryFromPath( ... ) on the results.
So, to determine the directory location of an application, you could do:
<cfset Application.Paths.Root = getDirectoryFromPath( getCurrentTemplatePath() ) />
Inside of the onApplicationStart event for your Application.cfc
To determine the path where the app server running your CFML engine is at, you can access shell commands with cfexecute, so (bearing in mind above discussions on pwd/etc) you can do:
Unix:
<cfexecute name="pwd"/>
for Windows, create a pwd.bat containing text #cd, then:
<cfexecute name="C:\docume~1\myuser\pwd.bat"/>
(Use the variable attribute of cfexecute to store the value instead of outputting to screen.)
In cmd (the Microsoft command line shell)
You can get the name of the script with %* (may be relative to pwd)
This gets directory of script:
set oldpwd=%cd%
cd %0\..
set app_dir=%pwd%
cd %oldpwd%
If you find any bugs, which you will. Then please fix or comment.
I released https://github.com/gpakosz/whereami which solves the problem in C and gives you:
the path to the current executable
the path to the current module (differs from path to executable when calling from a shared library).
It uses GetModuleFileNameW on Windows, parses /proc/self/maps on Linux and Android and uses _NSGetExecutablePath or dladdr on Mac and iOS.
Note to answer "20 above regarding Mac OSX only: If a JAR executable is transformed to an "app" via the OSX JAR BUNDLER, then the getClass().getProtectionDomain().getCodeSource().getLocation(); will NOT return the current directory of the app, but will add the internal directory structure of the app to the response. This internal structure of an app is /theCurrentFolderWhereTheAppReside/Contents/Resources/Java/yourfile
Perhaps this is a little bug in Java. Anyway, one must use method one or two to get the correct answer, and both will deliver the correct answer even if the app is started e.g. via a shortcut located in a different folder or on the desktop.
carl
SoundPimp.com