fix: handle non-ascii characters in page names and attachments

* Added Windows hints and v1.2 infos
* added some windows batch files
* switched from 'cliopts' to 'cli'
* "named args"
* All arguments passed to php are UTF-8 encoded
* removed local path from repository. Use doku.local.php
* added/changed support for
  - small
  - big
  - anchordef
  - definition_list
  - comment
  - include and search macros
* added #pragma directives as comments to dokuwiki
* recreate "_dokuwiki.changes"
* Support for non-ascii characters in media files

.gitignore

@@ -0,0 +1,4 @@
+settings.local.cmd
+*.log
+*.pyc
+out/

README

@@ -1,39 +0,0 @@
-Complete MoinMoin to DokuWiki converter
-
-Uses native MoinMoin modules to handle converting and translating paths.
-Converts also page history and edit-log.
-
-http://www.dokuwiki.org/tips:moinmoin2doku
-
-Tested with MoinMoin 1.5 and DokuWiki 2012-09-10 releases
-
-You need to run this on host where both MoinMoin and DokuWiki are configured,
-it uses current configuration from both wikis.
-
-Edit doku.php if your DokuWiki installation is other than /usr/share/dokuwiki
-
-To convert moinmoin all pages with history, invoke:
-$ ./moin2doku.py -a -d /var/lib/dokuwiki
-
-To convert single page (FrontPage):
-$ ./moin2doku.py -F moinmoin/data/pages/FrontPage -d out
-
-after conversion be sure to fix ownership
-(www-data:www-data being your uid/gid webserver runs):
-# chown -R www-data:www-data /var/lib/dokuwiki/pages/*
-# chown -R www-data:www-data /var/lib/dokuwiki/media/*
-
-also, depending on your configuration, you may need to gzip the attic pages.
-
-History:
-version 0.1  2010-02
-  Slim Gaillard, based on the "extended python" convert.py script here:
-  https://www.dokuwiki.org/tips:moinmoin2doku?rev=1297006559#extended_python
-
-version 0.2  2011
-  Elan Ruusamäe, moved to github, track history there
-  https://github.com/glensc/moin2doku
-
-version 1.0  2012
-  Complete moinmoin to dokuwiki converter, uses native moinmoin code to handle
-  converting and translating paths. Converts also page history and edit-log.

README.md

@@ -0,0 +1,93 @@
+Complete MoinMoin to DokuWiki converter
+=======================================
+
+Uses native MoinMoin modules to handle converting and translating paths.
+Converts also page history and edit-log.
+
+http://www.dokuwiki.org/tips:moinmoin2doku
+
+Tested with MoinMoin 1.9.9 and DokuWiki 2018-04-22 releases under Windows 7
+
+You need to run this on host where both MoinMoin and DokuWiki are configured,
+it uses current configuration from both wikis.
+
+Edit `doku.php` if your DokuWiki installation is other than `/usr/share/dokuwiki`
+
+To convert moinmoin all pages with history, invoke:
+```
+$ ./moin2doku.py -a -d /var/lib/dokuwiki
+```
+
+To convert single page (FrontPage):
+```
+$ ./moin2doku.py -F moinmoin/data/pages/FrontPage -d out
+```
+
+You should invoke `bin/indexer.php` after conversion to make all pages are indexed.
+
+and ensure ownership of files is correct:
+(`www-data:www-data` being your uid/gid webserver runs):
+```
+# chown -R www-data:www-data /var/lib/dokuwiki/pages/*
+# chown -R www-data:www-data /var/lib/dokuwiki/media/*
+```
+additionally, depending on your configuration, you may need to gzip the attic pages.
+
+
+Hints for Windows Users
+-----------------------
+
+The Batchfiles (`*.cmd`) should help to do the conversation under Windows. You should
+create a copy of the `settings.cmd` and call it `settings.local.cmd` to set your
+own local paths.
+
+Call `moin2doku.cmd` to convert the full MoinMoin Wiki. All DokuWiki pages will be
+written to an `out` folder in the current directory.
+
+This will convert a single page:
+```
+D:\moin2doku\> moin2doku.cmd MyMoinPage
+```
+
+Set `%OUTDIR%` to an alternativ output folder. This should not be the dokuwiki `data`
+folder if you want to do a full conversation.
+
+The `reindex.cmd` will call the `bin/indexer.php`-Skript.
+
+
+History
+=======
+
+version 0.1 (2010-02)
+-------------------
+
+Slim Gaillard, based on the "extended python" convert.py script here:
+https://www.dokuwiki.org/tips:moinmoin2doku?rev=1297006559#extended_python
+
+version 0.2 (2011)
+----------------
+
+Elan Ruusamäe, moved to github, track history there
+https://github.com/glensc/moin2doku
+
+version 1.0 (2012)
+----------------
+
+Complete moinmoin to dokuwiki converter, uses native moinmoin code to handle
+converting and translating paths. Converts also page history and edit-log.
+
+This marks the project "done", I will no longer develop it or support it, as I got my conversion done. However, I do accept patches (pull requests) to sane amount.
+
+I put repo online so others have better starting point than I did.
+
+
+version 1.1 (2015)
+----------------
+Modifed the script to work with newer Moin versions and API changes.
+
+
+version 1.2 (2019-01)
+----------------
+Some modifications to work with current DokuWiki and added more formattings.
+
+Search GitHub Forks for newer versions of this project.

doku.php

@@ -13,10 +13,14 @@
 
 if ('cli' != php_sapi_name()) die();
 
-define('DOKU_INC', '/usr/share/dokuwiki/');
+//add to following define of 'DOKU_INC' to your "doku.local.php" file and adjust the path:
+//define('DOKU_INC', '/home/caddy/wikifarm/dokuwiki/dokuwiki/');
+//define('DOKU_INC', "d:/website/wwwroot/dokuwiki/" );
+require_once './doku.local.php';
+
 require_once DOKU_INC.'inc/init.php';
 require_once DOKU_INC.'inc/common.php';
-require_once DOKU_INC.'inc/cliopts.php';
+require_once DOKU_INC.'inc/cli.php';
 
 # disable gzip regardless of config, then we don't have to compress when converting
 $conf['compression'] = 0; //compress old revisions: (0: off) ('gz': gnuzip) ('bz2': bzip)
@@ -29,29 +33,33 @@ function strip_dir($dir, $fn) {
 	return end(explode($dir.'/', $fn, 2));
 }
 
-switch ($argv[1]) {
+$action = $argv[1];
+$argPage = $argv[2];
+//filext = $argv[3];
+
+switch ($action) {
 case 'cleanID':
-	echo cleanID($argv[2]);
+	echo cleanID($argPage);
 	break;
 case 'wikiFN':
 	if ($argc > 3 && $argv[3]) {
-		echo strip_dir($conf['olddir'], wikiFN($argv[2], $argv[3]));
+		echo strip_dir($conf['olddir'], wikiFN($argPage, $argv[3]));
 	} else {
-		echo strip_dir($conf['datadir'], wikiFN($argv[2]));
+		echo strip_dir($conf['datadir'], wikiFN($argPage));
 	}
 	break;
 case 'mediaFN':
-	echo strip_dir($conf['mediadir'], mediaFN($argv[2]));
+	echo strip_dir($conf['mediadir'], mediaFN($argPage));
 	break;
 case 'metaFN':
-	echo strip_dir($conf['metadir'], metaFN($argv[2], $argv[3]));
+	echo strip_dir($conf['metadir'], metaFN($argPage, $argv[3]));
 	break;
 case 'getNS':
-	echo getNS($argv[2]);
+	echo getNS($argPage);
 	break;
 case 'getId':
 	echo getId();
 	break;
 default:
-	die("Unknown knob: {$argv[1]}");
+	die("Unknown knob: {$action}");
 }

doku.py

@@ -11,6 +11,9 @@
 import sys
 import subprocess
 
+from MoinMoin import log
+logging = log.getLogger(__name__)
+
 class DokuWiki:
 	def __init__(self):
 		self.callcache = {}
@@ -24,10 +27,17 @@ class DokuWiki:
 
 	def __call(self, method, *args):
 		args = list(args)
-		key = "%s:%s" % (method, ",".join(args))
+		uargs = []
+		for arg in args:
+			try:
+				arg.decode('utf-8')
+				#already UTF-8 ready
+				uargs.append(arg)
+			except UnicodeError:
+				uargs.append(arg.encode('utf-8'))
+		key = "%s:%s" % (method, ",".join(uargs))
 		if not self.callcache.has_key(key):
-			cmd = ['./doku.php', method ] + args
+			cmd = ['php', './doku.php', method] + [arg.encode("utf-8") for arg in uargs]
-			res = subprocess.Popen(cmd, stdin = None, stdout = subprocess.PIPE, stderr = sys.stderr, close_fds = True).communicate()
+			res = subprocess.Popen(cmd, stdin = None, stdout = subprocess.PIPE, stderr = sys.stderr, close_fds = False).communicate()
 			self.callcache[key] = unicode(res[0].decode('utf-8'))
-			print "%s->%s" % (cmd, self.callcache[key])
 		return self.callcache[key]

moin2doku.cmd

@@ -0,0 +1,36 @@
+@echo off
+setlocal
+
+call settings.cmd
+
+if not "%1"=="" goto :singlePage %1
+
+if "%OUTDIR%"=="" (
+  call :deldir out\attic || (pause & goto :eof)
+  call :deldir out\media || (pause & goto :eof)
+  call :deldir out\meta || (pause & goto :eof)
+  call :deldir out\pages || (pause & goto :eof)
+  if exist %~n0.pages.log del %~n0.pages.log
+  if not exist out md out
+  set OUTDIR=%CD%\out
+)
+
+call python moin2doku.py %DOKU_FULL_HISTORY% -d "%OUTDIR:\=/%" >"%~n0.log" 2>"%~n0.err.log"
+
+goto :eof
+:deldir
+if exist %1 rd /s/q %1
+if exist %1 exit /B 1
+
+goto :eof
+:singlePage
+if "%OUTDIR%"=="" set OUTDIR=%CD%\out
+call python moin2doku.py %DOKU_FULL_HISTORY% -p "%MOIN_DATA_HOME%\pages\%~1" -f -d "%OUTDIR:\=/%" >>"%~n0.log" 2>>"%~n0.err.log" || type "%~n0.err.log"
+if %ERRORLEVEL% == 0 if exist "%DOKU_ANIMALS_HOME%\%ANIMAL%\conf\local.php" (
+  rem touching "%DOKU_ANIMALS_HOME%\%ANIMAL%\conf\local.php" to invalidate cache
+  pushd "%DOKU_ANIMALS_HOME%\%ANIMAL%\conf"
+  copy /y/b local.php +,, >nul
+  popd
+)
+
+goto :eof

moin2doku.py

@@ -11,7 +11,7 @@
 import sys, os, os.path, re, codecs
 import getopt
 from MoinMoin import user, wikiutil
-from MoinMoin.request import RequestCLI
+from MoinMoin.web.contexts import ScriptContext as RequestCLI
 from MoinMoin.logfile import editlog
 from MoinMoin.Page import Page
 from shutil import copyfile, copystat
@@ -19,6 +19,11 @@ from os import listdir, mkdir
 from os.path import isdir, basename
 from doku import DokuWiki
 from moinformat import moin2doku
+import random
+
+# sys.setdefaultencoding() does not exist, here!
+reload(sys)  # Reload does the trick!
+sys.setdefaultencoding('cp1252')
 
 USEC = 1000000
 
@@ -40,8 +45,8 @@ def init_dirs(output_dir):
 		mkdir(metadir)
 
 def readfile(filename):
-	with open(filename, 'r') as f:
+	f = open(filename, 'r')
-		text = f.read()
+	text = f.read()
 	return unicode(text.decode('utf-8'))
 
 def writefile(filename, content, overwrite=False):
@@ -50,7 +55,7 @@ def writefile(filename, content, overwrite=False):
 		os.makedirs(dir);
 
 	if os.path.exists(filename) and overwrite == False:
-		raise OSError, 'File already exists: %s' % filename
+		raise (OSError, 'File already exists: %s' % filename)
 
 	# ensure it's a list
 	if not isinstance(content, (list, tuple)):
@@ -60,9 +65,14 @@ def writefile(filename, content, overwrite=False):
 	f.writelines([line + u'\n' for line in content])
 	f.close()
 
+
+def encode_relaxed(text):
+	return text.encode("ascii", errors="ignore")
+
+
 # page = MoinMoin Page oject
 # ns = DokuWiki namespace where attachments to copy
-def copy_attachments(page, ns):
+def copy_attachments(page, ns,randomID):
 	srcdir = page.getPagePath('attachments', check_create = 0)
 	if not isdir(srcdir):
 		return
@@ -73,10 +83,13 @@ def copy_attachments(page, ns):
 
 	attachments = listdir(srcdir)
 	for attachment in attachments:
-		src = os.path.join(srcdir, attachment)
+		try:
-		dst = os.path.join(output_dir, 'media', dw.mediaFN(dw.cleanID("%s/%s" % (ns, attachment))))
+			src = os.path.join(srcdir, attachment)
-		copyfile(src, dst)
+			dst = os.path.join(output_dir, 'media', dw.mediaFN(dw.cleanID(u"%s/%s" % (ns, str(randomID)+attachment))))
-		copystat(src, dst)
+			copyfile(src, dst)
+			copystat(src, dst)
+		except UnicodeDecodeError:
+			print 'ERROR: unable to convert attachment "%s"' % encode_relaxed(attachment)
 
 def print_help():
 	program = sys.argv[0]
@@ -156,13 +169,16 @@ def convert_editlog(page, output = None, overwrite = False):
 def convertfile(page, output = None, overwrite = False):
 	pagedir = page.getPagePath()
 	pagename = wikiname(pagedir)
+
 	if not output:
 		output = pagename
 
+	print "Converting %s" % encode_relaxed(pagename)
+
 	if page.isUnderlayPage():
 		print "underlay: %s" % page.request.cfg.data_underlay_dir
 		print "underlay: %s" % request.cfg.data_underlay_dir
-		print "SKIP UNDERLAY: %s" % pagename
+		print "SKIP UNDERLAY: %s" % encode_relaxed(pagename)
 		return False
 
 	current_exists = page.exists()
@@ -173,6 +189,9 @@ def convertfile(page, output = None, overwrite = False):
 	else:
 		revs = [current_rev]
 
+	# Generate random ID Number for collision avoidance when attachments in Namespace have the same name
+	randomID = random.randint(101,999)
+
 	for rev in revs:
 		page = Page(request, pagename, rev = rev)
 		pagefile, realrev, exists = page.get_rev(rev = rev);
@@ -188,7 +207,7 @@ def convertfile(page, output = None, overwrite = False):
 				print "recovered %s: %s" % (rev, mtime)
 
 			if not mtime:
-				print "NO REVISION: for %s" % pagefile
+				print "NO REVISION: for %s" % encode_relaxed(pagefile)
 				continue
 
 		if rev == current_rev:
@@ -199,7 +218,7 @@ def convertfile(page, output = None, overwrite = False):
 		else:
 			out_file = os.path.join(output_dir, 'attic', dw.wikiFN(output, str(mtime)))
 
-		content = moin2doku(pagename, page.get_raw_body())
+		content = moin2doku(pagename, page.get_raw_body(),randomID)
 		if len(content) == 0:
 #			raise Exception, "No content"
 			print "NO CONTENT: exists: %s,%s" % (exists, os.path.exists(pagefile))
@@ -208,7 +227,7 @@ def convertfile(page, output = None, overwrite = False):
 		copystat(pagefile, out_file)
 
 	ID = dw.cleanID(output)
-	copy_attachments(page, dw.getNS(ID))
+	copy_attachments(page, dw.getNS(ID),randomID)
 
 	# convert edit-log, it's always present even if current page is not
 	convert_editlog(page, output = output, overwrite = overwrite)
@@ -223,7 +242,7 @@ def convertfile(page, output = None, overwrite = False):
 		if old_page != ID:
 			redirect_map[old_page] = ID
 
-	print "Converted %s as %s" % (pagename, dw.wikiFN(output))
+	print "Converted %s as %s" % (encode_relaxed(pagename), dw.wikiFN(output))
 
 	return True
 
@@ -294,6 +313,7 @@ else:
 	# get list of all pages in wiki
 	# hide underlay dir temporarily
 	underlay_dir = request.rootpage.cfg.data_underlay_dir
+	print(underlay_dir)
 	request.rootpage.cfg.data_underlay_dir = None
 	pages = request.rootpage.getPageList(user = '', exists = not convert_attic, filter = filter)
 	pages = dict(zip(pages, pages))
@@ -307,6 +327,11 @@ else:
 		del pages[frontpage.page_name]
 	pages[dw.getId()] = frontpage.page_name
 
+print "--------------------------------------------------"
+for output, pagename in pages.items():
+	print " - %s" % encode_relaxed(pagename)
+print "--------------------------------------------------"
+
 converted = 0
 for output, pagename in pages.items():
 	page = Page(request, pagename)

moinformat.py

@@ -7,20 +7,22 @@
 #
 # Author: Elan Ruusamäe <glen@pld-linux.org>
 # Version: 1.0
-
+#from MoinMoin.web.request import Request as RequestCLI
-from MoinMoin import wikimacro, wikiutil
+from MoinMoin.web.contexts import ScriptContext
+from MoinMoin import  wikiutil
 from MoinMoin.Page import Page
-from MoinMoin.parser.wiki import Parser
+from MoinMoin.parser.text_moin_wiki import Parser
 
 from text_dokuwiki import Formatter
-from MoinMoin.request import RequestCLI
 
 import sys
 import StringIO
 
-def moin2doku(pagename, text):
+def moin2doku(pagename, text, randomID=None):
 	parser = Parser(text, request)
 
+	formatter.setRandomID(randomID)
+	
 	# this needed for macros
 	request.formatter = formatter
 
@@ -37,7 +39,7 @@ def moin2doku(pagename, text):
 
 	return unicode(output.getvalue().decode('utf-8'))
 
-request = RequestCLI()
+request = ScriptContext()
 formatter = Formatter(request)
 
 if __name__ == "__main__":
@@ -47,6 +49,6 @@ if __name__ == "__main__":
 	else:
 		inputfile = 'syntaxreference.txt'
 
-	with open(inputfile, 'r') as f:
+	f = open(inputfile, 'r')
-		text = f.read()
+	text = f.read()
 	print moin2doku('test', text)

reindex.cmd

@@ -0,0 +1,69 @@
+@echo off
+setlocal
+
+call settings.cmd
+
+pushd "%DOKU_ANIMALS_HOME%\%ANIMAL%"
+
+if not exist data\pages\ (
+  echo WARNUNG:
+  echo Es wurden kein "pages" Verzeichnis gefunden. Wurde die neue Version bereits
+  echo ins Zielverzeichnis kopiert?
+  pause
+  goto :eof
+)
+
+if not exist data\media\logo.png (
+  echo working in %CD%
+  rem call :cleanup data\attic || goto :ende
+  call :cleanup data\cache || goto :ende
+  call :cleanup data\index || goto :ende
+  call :cleanup data\locks || goto :ende
+  call :cleanup data\media_attic || goto :ende
+  call :cleanup data\media_meta || goto :ende
+  call :cleanup data\tmp || goto :ende
+  xcopy /S/I/Y/Q common\*.* data
+)
+
+REM if exist data\pages\startseiteneu.txt (
+  REM ren data\pages\startseiteneu.txt startseite.txt
+  REM ren data\meta\startseiteneu.changes startseite.changes
+REM )
+
+popd 
+pushd "%DOKU_HOME%"
+
+php bin\indexer.php || goto :ende
+
+popd
+pushd "%DOKU_ANIMALS_HOME%\%ANIMAL%"
+
+if exist data\meta\_dokuwiki.changes del data\meta\_dokuwiki.changes
+if exist data\meta\_dokuwiki.changes del data\meta\_dokuwiki.changes
+(
+  for /F "delims=*" %%D in ('dir /b/S/A:D data\meta\*.*') do (
+    if exist "%%D\*.changes" type "%%D\*.changes" 2>nul
+  )
+) > _dokuwiki_unsorted.changes
+sort _dokuwiki_unsorted.changes /O data\meta\_dokuwiki.changes
+del _dokuwiki_unsorted.changes
+
+echo --- compressing old files
+for /F "delims=*" %%T in ('dir data\attic\*.txt /s/b') do (
+  "c:\Program Files\7-Zip\7z.exe" a -bso0 "%%T.gz" "%%T" && del "%%T" || goto :ende
+)
+echo --- done
+
+:ende
+popd
+
+pause
+
+goto :eof
+:cleanup
+if not exist %1 goto :eof
+echo cleaning up %1
+rd /s/q %1
+if exist %1 exit /b 1
+md %1
+echo >nul 2>"%~1\_dummy"

settings.cmd

@@ -0,0 +1,45 @@
+@echo off
+REM -- no setlocal in this script!
+REM make a copy of this file and adjust the paths
+
+set PHP_HOME=c:\Program Files\php
+set PYTHON_HOME=c:\Python27
+REM -- MoinMoin settings 
+set MOIN_HOME=c:\wwwroot\wiki\moin
+set MOIN_CONFIG=%MOIN_HOME%\wiki\config
+REM set MOIN_CONFIG=c:\wwwroot\moinfarmdata\config
+set MOIN_DATA_HOME=%MOIN_HOME%\wiki\data
+REM set MOIN_CONFIG=c:\wwwroot\moinfarmdata\<farmwiki>
+REM -- DokuWiki settings
+set DOKU_HOME=c:\wwwroot\wiki\dokuwiki
+set DOKU_ANIMALS_HOME=%DOKU_HOME%
+REM set DOKU_ANIMALS_HOME=c:\wwwroot\dokufarmdata
+REM set animal=<yourFarmAnimalName>
+REM comment this in to do a full converstion
+REM set DOKU_FULL_HISTORY=-a
+
+REM -- path to your php.ini used by your webserver
+REM set PHP_INI_SCAN_DIR=c:\Program Files\ApacheHttpd\conf\
+
+REM -- set this to your "production" dokuwiki if you want to update only.
+REM set OUTDIR=%DOKU_ANIMALS_HOME%\%animal%\data
+
+REM ----8<--------8<--------8<--------8<--------8<--------8<--------8<----
+REM -- remove everything beyond the line from your copy
+chcp 1252
+
+if exist "%~dpn0.local.cmd" call "%~dpn0.local.cmd"
+
+REM -- %animal% must be lowercase!
+(
+set animal=
+set animal=%animal%
+)
+
+set PATH=%PATH%;%PYTHON_HOME%;%PHP_HOME%
+
+set PYTHONPATH=.
+set PYTHONPATH=%PYTHONPATH%;%MOIN_HOME%
+set PYTHONPATH=%PYTHONPATH%;%MOIN_HOME%\MoinMoin\support
+set PYTHONPATH=%PYTHONPATH%;%MOIN_CONFIG%
+set PYTHONPATH=%PYTHONPATH%;

text_dokuwiki.py

@@ -1,346 +1,537 @@
 # -*- coding: utf-8 -*-
 # Setup VIM: ex: noet ts=2 sw=2 :
 """
-	MoinMoin - Dokuwiki Formatter
+    MoinMoin - Dokuwiki Formatter
 
-	@copyright: 2000, 2001, 2002 by Jürgen Hermann <jh@web.de>
+    @copyright: 2000, 2001, 2002 by Jürgen Hermann <jh@web.de>
-	@copyright: 2011-2012 Elan Ruusamäe <glen@delfi.ee>
+    @copyright: 2011-2012 Elan Ruusamäe <glen@delfi.ee>
-	@license: GNU GPL, see COPYING for details.
+    @license: GNU GPL, see COPYING for details.
 """
 
 from xml.sax import saxutils
-from MoinMoin.formatter.base import FormatterBase
+from MoinMoin.formatter import FormatterBase
 from MoinMoin import config
 from MoinMoin.Page import Page
 from types import *
+from MoinMoin import log
+
+logging = log.getLogger(__name__)
 
 # TODO: let base class MoinMoin/formatter/base.py handle not implemented methods
 
 class Formatter(FormatterBase):
-	"""
+    """
-		Send Dokuwiki formatted data.
+        Send Dokuwiki formatted data.
-	"""
+    """
-
+
-	hardspace = '&nbsp;'
+    hardspace = '&nbsp;'
-#	hardspace = ' '
+#   hardspace = ' '
-
+
-	def __init__(self, request, **kw):
+    def __init__(self, request, **kw):
-		apply(FormatterBase.__init__, (self, request), kw)
+        apply(FormatterBase.__init__, (self, request), kw)
-		self._current_depth = 1
+        self._current_depth = 1
-		self._base_depth = 0
+        self._base_depth = 0
-		self.in_pre = 0
+        self.in_pre = 0
-		self.in_table = 0
+        self.in_table = 0
-		self._text = None # XXX does not work with links in headings!!!!!
+        self._text = None # XXX does not work with links in headings!!!!!
-
+        self.randomID= None
-		self.list_depth = 0
+        self.list_depth = 0
-		self.list_type = ' '
+        self.list_type = ' '
-
+
-	def _escape(self, text, extra_mapping={"'": "&apos;", '"': "&quot;"}):
+    def setRandomID(self,ID):
-		return saxutils.escape(text, extra_mapping)
+        self.randomID = str(ID)
-
+
-	def startDocument(self, pagename):
+    def _escape(self, text, extra_mapping={"'": "&apos;", '"': "&quot;"}):
-		encoding = config.charset
+        return saxutils.escape(text, extra_mapping)
-		return '<?xml version="1.0" encoding="%s"?>\n<s1 title="%s">' % (
+
-			encoding, self._escape(pagename))
+    def startDocument(self, pagename):
-
+        encoding = config.charset
-	def endDocument(self):
+        return '<?xml version="1.0" encoding="%s"?>\n<s1 title="%s">' % (
-		result = ""
+            encoding, self._escape(pagename))
-		while self._current_depth > 1:
+
-			result += "</s%d>" % self._current_depth
+    def endDocument(self):
-			self._current_depth -= 1
+        result = ""
-		return result + '</s1>'
+        while self._current_depth > 1:
-
+            result += "</s%d>" % self._current_depth
-	def lang(self, on, lang_name):
+            self._current_depth -= 1
-		return ('<div lang="">' % lang_name, '</div>')[not on]
+        return result + '</s1>'
-
+
-	def sysmsg(self, on, **kw):
+    def lang(self, on, lang_name):
-		return ('<sysmsg>', '</sysmsg>')[not on]
+        return ('<div lang="">' % lang_name, '</div>')[not on]
-
+
-	def rawHTML(self, markup):
+    def sysmsg(self, on, **kw):
-		return '<html>' + markup + '</html>'
+        return ('<sysmsg>', '</sysmsg>')[not on]
-
+
-	def pagelink(self, on, pagename='', page=None, **kw):
+    def rawHTML(self, markup):
-		if on:
+        #return '<html>' + markup + '</html>'
-			return '[[:' + ":".join(pagename.split("/")) + "|"
+        return ''
-		else:
+
-			return ']]'
+    def pagelink(self, on, pagename='', page=None, **kw):
-
+        if on:
-	def interwikilink(self, on, interwiki='', pagename='', **kw):
+            return '[[:' + ":".join(pagename.split("/")) + "|"
-		if on:
+        else:
-			if interwiki == 'Self':
+            return ']]'
-				return self.pagelink(on, pagename, **kw)
+
-			return '[[%s>%s|' % (interwiki, pagename)
+    def interwikilink(self, on, interwiki='', pagename='', **kw):
-		else:
+        if on:
-			return ']]'
+            if interwiki == 'Self':
-
+                return self.pagelink(on, pagename, **kw)
-	def url(self, on, url='', css=None, **kw):
+            interwikis = {
-		return ('[[%s|' % (self._escape(url)), ']]')[not on]
+                'WikiPedia':'wp',
-
+                'FrWikiPedia':'wpfr',
-	def attachment_link(self, url, text, **kw):
+                'DeWikiPedia':'wpde',
-		return '{{%s|%s}}' % (url, text)
+                'MetaWikiPedia':'wpmeta'
-
+                }
-	def attachment_image(self, url, **kw):
+            if interwiki in interwikis:
-		return '{{%s|}}' % (url,)
+                return '[[%s>%s|' % (interwikis.get(interwiki), pagename)
-
+            return '[[%s>%s|' % (interwiki, pagename)
-	def attachment_drawing(self, url, text, **kw):
+        else:
-		return '{{%s|%s}}' % (url, text)
+            return ']]'
-
+
-	def text(self, text, **kw):
+    def url(self, on, url='', css=None, **kw):
-		self._did_para = 0
+        return ('[[%s|' % (self._escape(url)), ']]')[not on]
-		if self._text is not None:
+
-			self._text.append(text)
+    def attachment_link(self, on, url=None, querystr=None, **kw):
-		return text
+        if on:
-
+            return '{{ %s | ' % (self.randomID+url)
-	def rule(self, size=0, **kw):
+        else: 
-		# size not supported
+            return ' }}'
-		if size >= 4:
+
-			return '----\n'
+    def attachment_image(self, url, **kw):
-		else:
+        return '{{%s|}}' % (self.randomID+url,)
-			return '-' * size + '\n'
+
-
+    def attachment_drawing(self, url, text, **kw):
-	def icon(self, type):
+        return '{{%s|%s}}' % (self.randomID+url, text)
-		return '<icon type="%s" />' % type
+
-
+    def text(self, text, **kw):
-	def strong(self, on, **kw):
+        self._did_para = 0
-		return ['**', '**'][not on]
+        if self._text is not None:
-
+            self._text.append(text)
-	def emphasis(self, on, **kw):
+        return text
-		return ['//', '//'][not on]
+
-
+    def rule(self, size=0, **kw):
-	def highlight(self, on, **kw):
+        # size not supported
-		return ['**', '**'][not on]
+        if size >= 4:
-
+            return '----\n'
-	def number_list(self, on, type=None, start=None, **kw):
+        else:
-		# list type not supported
+            return '-' * size + '\n'
-		if on:
+
-			self.list_depth += 1
+    def icon(self, type):
-			self.list_type = '-'
+        return '<icon type="%s" />' % type
-		else:
+
-			self.list_depth -= 1
+    def strong(self, on, **kw):
-			self.list_type = ' '
+        return ['**', '**'][not on]
-
+
-		return ['', '\n'][on]
+    def emphasis(self, on, **kw):
-
+        return ['//', '//'][not on]
-	def bullet_list(self, on, **kw):
+
-		if on:
+    def highlight(self, on, **kw):
-			self.list_depth += 1
+        return ['**', '**'][not on]
-			self.list_type = '*'
+
-		else:
+    def number_list(self, on, type=None, start=None, **kw):
-			self.list_depth -= 1
+        # list type not supported
-			self.list_type = ' '
+        if on:
-
+            self.list_depth += 1
-		return ['', '\n'][on]
+            self.list_type = '-'
-
+        else:
-	def listitem(self, on, **kw):
+            self.list_depth -= 1
-		# somewhy blockquote uses "listitem" call
+            self.list_type = ' '
-		return [(' ' * self.list_depth * 2) + self.list_type + ' ', '\n'][not on]
+
-
+        return ['', '\n'][on]
-	def code(self, on, **kw):
+
-		""" `typewriter` or {{{typerwriter}}, for code blocks i hope codeblock works """
+    def bullet_list(self, on, **kw):
-		return ["''", "''"][not on]
+        if on:
-
+            self.list_depth += 1
-	def sup(self, on, **kw):
+            self.list_type = '*'
-		return ['<sup>', '</sup>'][not on]
+        else:
-
+            self.list_depth -= 1
-	def sub(self, on, **kw):
+            if self.list_depth <= 0:
-		return ['<sub>', '</sub>'][not on]
+                self.list_type = ' '
-
+
-	def strike(self, on, **kw):
+        return ['', '\n'][on]
-		return ['<del>', '</del>'][not on]
+
-
+    # generic transclude/include:
-	def preformatted(self, on, **kw):
+    def transclusion(self, on, **kw):
-		FormatterBase.preformatted(self, on)
+        return ''
-		result = ''
+
-		if self.in_p:
+    def transclusion_param(self, **kw):
-			result = self.paragraph(0)
+        return ''
-		return result + ['<file>', '</file>\n'][not on]
+
-
+    def listitem(self, on, **kw):
-	def paragraph(self, on, **kw):
+        # somewhy blockquote uses "listitem" call
-		FormatterBase.paragraph(self, on)
+        return [(' ' * self.list_depth * 2) + self.list_type + ' ', '\n'][not on]
-		if self.in_table or self.list_depth:
+
-			return ''
+    def code(self, on, **kw):
-		return ['', '\n\n'][not on]
+        """ `typewriter` or {{{typerwriter}}, for code blocks i hope codeblock works """
-
+        return ["''%%", "%%''"][not on]
-	def linebreak(self, preformatted=1):
+
-		return ['\n', '\\\n'][not preformatted]
+    def sup(self, on, **kw):
-
+        return ['<sup>', '</sup>'][not on]
-	def heading(self, on, depth, **kw):
+
-		# heading depth reversed in dokuwiki
+    def sub(self, on, **kw):
-		heading_depth = 7 - depth
+        return ['<sub>', '</sub>'][not on]
-
+
-		if on:
+    def strike(self, on, **kw):
-			return u'%s ' % (u'=' * heading_depth)
+        return ['<del>', '</del>'][not on]
-		else:
+
-			return u' %s\n' % (u'=' * heading_depth)
+    def small(self, on, **kw):
-
+        #https://www.dokuwiki.org/plugin:wrap
-	def table(self, on, attrs={}, **kw):
+        return ['<wrap lo>', '</wrap>'][not on]
-		if on:
+
-			self.in_table = 1
+    def big(self, on, **kw):
-		else:
+        #https://www.dokuwiki.org/plugin:wrap
-			self.in_table = 0
+        return ['<wrap hi>', '</wrap>'][not on]
-		return ''
+
-
+    def preformatted(self, on, **kw):
-	def table_row(self, on, attrs={}, **kw):
+        FormatterBase.preformatted(self, on)
-		return ['\n', '|'][not on]
+        result = ''
-
+        if self.in_p:
-	def table_cell(self, on, attrs={}, **kw):
+            result = self.paragraph(0)
-		return ['|', ''][not on]
+        return result + ['<code>', '</code>\n'][not on]
-
+
-	def anchordef(self, id):
+    def paragraph(self, on, **kw):
-		# not supported
+        FormatterBase.paragraph(self, on)
-		return ''
+        if self.in_table or self.list_depth:
-
+            return ''
-	def anchorlink(self, on, name='', **kw):
+        return ['', '\n\n'][not on]
-		# kw.id not supported, we hope the anchor matches existing heading on page
+
-		return ('[[#', ']]') [not on]
+    def linebreak(self, preformatted=1):
-
+        return ['\n', '\\\n'][not preformatted]
-	def underline(self, on, **kw):
+
-		return ['__', '__'][not on]
+    def heading(self, on, depth, **kw):
-
+        # heading depth reversed in dokuwiki
-	def definition_list(self, on, **kw):
+        heading_depth = 7 - depth
-		result = ''
+
-		if self.in_p:
+        if on:
-			result = self.paragraph(0)
+            return u'%s ' % (u'=' * heading_depth)
-		return result + ['<gloss>', '</gloss>'][not on]
+        else:
-
+            return u' %s\n' % (u'=' * heading_depth)
-	def definition_term(self, on, compact=0, **kw):
+
-		return ['<label>', '</label>'][not on]
+    def table(self, on, attrs={}, **kw):
-
+        if on:
-	def definition_desc(self, on, **kw):
+            self.in_table = 1
-		return ['<item>', '</item>'][not on]
+        else:
-
+            self.in_table = 0
-	def image(self, src=None, **kw):
+        return ['', '\n'][not on]
-		valid_attrs = ['src', 'width', 'height', 'alt', 'title']
+
-
+    def table_row(self, on, attrs={}, **kw):
-		url = src
+        return ['\n', '|'][not on]
-		if '?' in url:
+
-			url += '&'
+    def table_cell(self, on, attrs={}, **kw):
-		else:
+        return ['|', ''][not on]
-			url += '?'
+
-
+    def anchordef(self, id):
-		attrs = {}
+        # https://www.dokuwiki.org/plugin:anchor
-		for key, value in kw.items():
+        return '{{anchor:'+id+'}}'
-			if key in valid_attrs:
+
-				attrs[key] = value
+    def anchorlink(self, on, name='', **kw):
-
+        # kw.id not supported, we hope the anchor matches existing heading on page
-		# TODO: finish this
+        return ('[[#', ']]') [not on]
-		if attrs.has_key('width'):
+
-			url += attrs['width']
+    def underline(self, on, **kw):
-
+        return ['__', '__'][not on]
-		return '{{' + url + '}}'
+
-
+    def definition_list(self, on, **kw):
-	def code_area(self, on, code_id, code_type='code', show=0, start=-1, step=-1):
+        # https://www.dokuwiki.org/plugin:definitionlist
-		syntax = ''
+        result = ''
-		# switch for Python: http://simonwillison.net/2004/may/7/switch/
+        if self.in_p:
-		try:
+            result = self.paragraph(0)
-			syntax = {
+        return result
-				'ColorizedPython': 'python',
+
-				'ColorizedPascal': 'pascal',
+    def definition_term(self, on, compact=0, **kw):
-				'ColorizedJava': 'java',
+		#MoinMoin does no wiki markup in DL-Terms
-				'ColorizedCPlusPlus': 'cpp',
+        return ['  ;%%', '%%\n'][not on]
-			}[code_type]
+
-		except KeyError:
+    def definition_desc(self, on, **kw):
-			pass
+        return ['  :', '\n'][not on]
-
+
-		return ('<code %s>' % syntax , '</code>')[not on]
+    def image(self, src=None, **kw):
-
+        valid_attrs = ['src', 'width', 'height', 'alt', 'title']
-	def code_line(self, on):
+
-		return ('', '\n')[on]
+        url = src
-
+        if '?' in url:
-	def code_token(self, on, tok_type):
+            url += '&'
-		# not supported
+        else:
-		return ''
+            url += '?'
-
+
-	def comment(self, text):
+        attrs = {}
-		# real comments (lines with two hash marks)
+        for key, value in kw.items():
-		if text[0:2] == '##':
+            if key in valid_attrs:
-			return "/* %s */\n" % text[2:].strip()
+                attrs[key] = value
-
+
-		# Some kind of Processing Instruction
+        # TODO: finish this
-		# http://moinmo.in/HelpOnProcessingInstructions
+        if attrs.has_key('width'):
-		tokens = text.lstrip('#').split(None, 1)
+            url += attrs['width']
-		if tokens[0] in ('language', 'format', 'refresh'):
+
-			return ''
+        return '{{' + url + '}}'
-
+
-		if tokens[0] == 'acl':
+    def code_area(self, on, code_id, code_type='code', show=0, start=-1, step=-1,msg=None):
-			# TODO: fill acl.auth.php
+        syntax = ''
-			return ''
+        # switch for Python: http://simonwillison.net/2004/may/7/switch/
-
+        try:
-		if tokens[0] == 'deprecated':
+            syntax = {
-			return '<note warning>This page is deprecated</note>\n'
+                'ColorizedPython': 'python',
-
+                'ColorizedPascal': 'pascal',
-		if tokens[0] == 'redirect':
+                'ColorizedJava': 'java',
-			return text + "\n"
+                'ColorizedCPlusPlus': 'cpp',
-
+            }[code_type]
-		if tokens[0] == 'pragma':
+        except KeyError:
-			# TODO: can do 'description' via 'meta' dokuwiki plugin
+            pass
-			return "/* pragma: %s */\n" % " ".join(tokens[1:])
+
-
+        return ('<code %s>' % syntax , '</code>')[not on]
-		return "/* %s */\n" % text.lstrip('#')
+
-
+    def code_line(self, on):
-	def macro(self, macro_obj, name, args):
+        return ('', '\n')[on]
-		def email(args):
+
-			mail = args.replace(' AT ', '@')
+    def code_token(self, on, tok_type):
-			mail = mail.replace(' DOT ', '.')
+        # not supported
-			return '[[%s|%s]]' % (mail, args)
+        return ''
-
+
-		# function which will just do what parent class would
+    def comment(self, text):
-		def inherit(args):
+        # real comments (lines with two hash marks)
-			return apply(FormatterBase.macro, (self, macro_obj, name, args))
+        if text[0:2] == '##':
-
+            # https://www.dokuwiki.org/plugin:comment
-		try:
+            comment = text[2:].strip()
-			lookup = {
+            if len(comment)>1:
-				'BR' : '\\\\',
+                return "/* %s */\n" % text[2:].strip()
-				'MailTo' : email,
+            return '\n'
-				'GetText' : args,
+
-				'ShowSmileys' : inherit,
+        # Some kind of Processing Instruction
-			}[name]
+        # http://moinmo.in/HelpOnProcessingInstructions
-		except KeyError:
+        tokens = text.lstrip('#').split(None, 1)
-			lookup = '/* UndefinedMacro: %s(%s) */' % (name, args)
+        if tokens[0] in ('language', 'format', 'refresh'):
-
+            return ''
-		if type(lookup) == FunctionType:
+
-			text = lookup(args)
+        if tokens[0] == 'acl':
-		else:
+            # TODO: fill acl.auth.php
-			text = lookup
+            logging.info('SKIPPING ACL: %s', text)
-		return text
+            return ''
-
+
-	def smiley(self, text):
+        if tokens[0] == 'deprecated':
-		try:
+            return '<note warning>This page is deprecated</note>\n'
-			# https://www.dokuwiki.org/devel:smileys.conf
+
-			return {
+        if tokens[0] == 'redirect':
-				# note: reverse sorted so that longer smileys get matched first
+            return text + "\n"
-				'X-('   : ':-X',
+
-				'{X}'   : ':!:',
+        if tokens[0] == 'pragma':
-				'{*}'   : '<ubu>',
+            # TODO: can do 'description' via 'meta' dokuwiki plugin
-				'(./)'  : u'✓',
+            pargs = tokens[1].split(None, 1)
-				':))'   : ':-P',
+            if pargs[0]=='section-numbers':
-				':-))'  : ':-P',
+                return '/* meta: %s */' % tokens
-				':-?'   : ':-P',
+            logging.info('SKIPPING PRAGMA: %s', tokens)
-				':o'    : ':-o',
+            #return "/* pragma: %s */\n" % " ".join(tokens[1:])
-				'{OK}'  : ':!:',
+            return ''
-				'{o}'   : '<circ>',
+
-				'{i}'   : ':!:',
+        #return "/* %s */\n" % text.lstrip('#')
-				':D'    : ':-D',
+        return ''
-				'B)'    : '8-)',
+
-				'B-)'   : '8-)',
+
-				'{3}'   : '<3>',
+    def macro(self, macro_obj, name, args,markup):
-				'{2}'   : '<2>',
+        def email(args):
-				'{1}'   : '<1>',
+            mail = args.replace(' AT ', '@')
-				'(!)'   : ':!:',
+            mail = mail.replace(' DOT ', '.')
-				'/!\\'  : ':!:',
+            return '[[%s|%s]]' % (mail, args)
-				':\\'   : ':-\\',
+
-				':))'   : ':-)',
+        def showAttachedFiles(args):
-				':)'    : ':-)',
+            args = args.split(',')
-				':('    : ':-(',
+            if len(args)>1:
-				':-))'  : ':-)',
+                return '{{ %s | %s }}' % (self.randomID+args[0].strip(), args[1].strip())
-				':-)'   : ':-)',
+            else:
-				':-('   : ':-(',
+                return ''
-				';)'    : ';-)',
+
-				'|)'    : ':-|',
+        # function which will just do what parent class would
-				'|-)'   : ':-|',
+        def inherit(args):
-				'>:>'   : '^_^',
+            return apply(FormatterBase.macro, (self, macro_obj, name, args))
-				'<!>'   : ':!:',
+
-				'<:('   : ':-?',
+        def randomQuote(args):
-			}[text]
+            # https://www.dokuwiki.org/plugin:xfortune
-		except KeyError:
+            return '{{xfortune>quote:'+args+'.txt}}'
-			return text
+
+        def monthcal(args):
+            # https://www.dokuwiki.org/plugin:monthcal
+            selfname = self.page.page_name
+            return '{{monthcal:create_links=short,namespace='+selfname.replace('/',':')+'}}'
+
+        def navigation(args):
+            # https://www.dokuwiki.org/plugin:alphaindex
+            selfname = self.page.page_name
+            args = args.split(',')
+            if len(args)>0:
+                try:
+                    result = {
+                      'slides': '[<>]',
+                      'children': '{{alphaindex>:%s#1|nons incol}}' % selfname.replace('/',':'),
+                      'siblings': '{{alphaindex>.#1|nons incol}}',
+                      'slideshow': '/* no support for slideshow navigation */'
+                    }[args[0].strip()]
+                except KeyError:
+                    result = '/* Unknown Navigation: %s #%s#*/' % args, args[0].strip()
+            else:
+                result = '/* Unsupported Navigation: %s */' % args
+            return result
+
+        def footnote(args):
+            return '((%s))' % args
+
+        def dateTimeMacro(args):
+            #https://www.dokuwiki.org/plugin:date
+            #args = args.split(',');
+            return '{{date>%%c|timestamp=strtotime("%s")|locale=de}}' % args
+
+        def dateMacro(args):
+            #https://www.dokuwiki.org/plugin:date
+            #args = args.split(',');
+            return '{{date>%%x|timestamp=strtotime("%s")|locale=de}}' % args
+
+        def includeMacro(args):
+            #https://www.dokuwiki.org/plugin:include
+            #logging.info('Include(%s)' % args)
+            args = map(unicode.strip, args.split(','));
+            #dokupage = ":".join(pagename.split("/"))
+            if len(args)==1:
+                return '{{page>%s&nodate}}' % ":".join(args[0].split("/"))
+            elif(u'titlesonly' in args):
+                #https://www.dokuwiki.org/plugin:changes
+                #https://www.dokuwiki.org/plugin:pagelist
+                selfname = self.page.page_name
+                selfNs = ":".join(selfname.split("/")).lower()
+                pairs = [arg.split('=') for arg in args]
+                # attrs = {}
+                # for key, value in pairs:
+                    # attrs[key] = value
+                #logging.info('pairs:"%s"' % pairs)
+
+                incName = ''#pairs[0]
+                incCount = -1
+                incTitlesOnly = False
+                notNamedParam = 0
+                for pair in pairs:
+                    if len(pair)==1:
+                        if u'titlesonly'==pair:
+                            notNamedParam = -1
+                            incTitlesOnly = True
+                        elif notNamedParam >=0:
+                            if notNamedParam==0:
+                                incName = pairs[notNamedParam]
+                            notNamedParam += 1;
+                    else:
+                        notNamedParam = -1
+                        if u'items'==pair[0]:
+                            incCount = int(pair[1])
+
+                resultArgs = '-h1 -textPages=""'
+                #(keys,values) = map()
+                if incCount > 0:
+                    resultArgs += ' -idAndTitle -simpleList -sortId -nbItemsMax=%d' % incCount
+                else:
+                    resultArgs += ' -nbCol=2'
+
+                ##<nspages fortran:mailarchiv -pregPagesOn="/2.*/" -h1 -nbCol=2 -textPages="">
+                ## Lister der letzten 10 Mails:
+                ##<nspages .:mailarchiv -nbItemsMax=10 -sortId -simpleList -idAndTitle -reverse -h1 -nbCol=1 -textPages=""> 
+                if incName[0]=='^':
+                    nspagedelim = incName.rfind('/')
+                    ns = ":".join(incName[1:nspagedelim].split('/')).lower()
+                    incPageReg = incName[(nspagedelim+1):]
+                    resultArgs += ' -pregPagesOn="/^%s/"' % incPageReg
+                else:
+                    ns = selfNs
+
+                return '<nspages %s %s>' % (ns, resultArgs)
+
+            else:
+                logging.info('UNSUPPORTED INCLUDE "%s"' % args)
+                return '/* Unsupported Include: %s */' % args
+
+        def fullsearch(args):
+            #args=None >> {searchform ns=}
+            #args='' >> {{backlinks>.}}
+            #args!='' >> {{search><args>}}
+            #ignore special searches. see MoinMoin page "HilfeZumSuchen"
+            if args is None:
+                return '{searchform ns=}'
+            elif ':' in args or ' ' in args:
+                logging.info('UNSUPPORTED SEARCH %s(%s)' % (name, args))
+                return '/* Unsupported Search %s(%s). may be backlinks plugin will help */' % (name, args)
+            elif args=='':
+                return '{{backlinks>.}}'
+            elif name=='PageList':
+                return '{{backlinks>%s}}' % ":".join(args.split('/')).lower();
+            else:
+                logging.info('UNSUPPORTED SEARCH %s(%s)' % (name, args))
+                return '/* Unsupported Search %s(%s) */' % (name, args)
+
+        try:
+            lookup = {
+                'BR' : ' \\\\ ',
+                'br' : ' \\\\ ',
+                'MailTo' : email,
+                'GetText' : args,
+                'ShowSmileys' : inherit,
+                'ShowAttachedFiles' : showAttachedFiles,
+                'Include' : includeMacro,
+                #no real fulltext search!
+                'FullSearch' : fullsearch,
+                'FullSearchCached' : fullsearch,
+                'PageList' : fullsearch,
+                'MonthCalendar' : monthcal,
+                'Navigation' : navigation,
+                'TableOfContents' : '',
+                'RandomQuote': randomQuote,
+                'Anchor': inherit,
+                'Action': inherit,
+                'Icon': inherit,
+                'FootNote': footnote,
+                'Date': dateMacro,
+                'DateTime': dateTimeMacro
+            }[name]
+        except KeyError:
+            logging.info('UNDEFINED MACRO "%s"' % name)
+            lookup = '/* UndefinedMacro: %s(%s) */' % (name, args)
+
+        if type(lookup) == FunctionType:
+            text = lookup(args)
+        else:
+            text = lookup
+        return text
+
+    def smiley(self, text):
+        try:
+            # https://www.dokuwiki.org/devel:smileys.conf
+            return {
+                # note: reverse sorted so that longer smileys get matched first
+                'X-('   : ':-X',
+                '{X}'   : ':!:',
+                '{*}'   : '<ubu>',
+                '(./)'  : u'✓',
+                ':))'   : ':-P',
+                ':-))'  : ':-P',
+                ':-?'   : ':-P',
+                ':o'    : ':-o',
+                '{OK}'  : ':!:',
+                '{o}'   : '<circ>',
+                '{i}'   : ':!:',
+                ':D'    : ':-D',
+                'B)'    : '8-)',
+                'B-)'   : '8-)',
+                '{3}'   : '<3>',
+                '{2}'   : '<2>',
+                '{1}'   : '<1>',
+                '(!)'   : ':!:',
+                '/!\\'  : ':!:',
+                ':\\'   : ':-\\',
+                ':))'   : ':-)',
+                ':)'    : ':-)',
+                ':('    : ':-(',
+                ':-))'  : ':-)',
+                ':-)'   : ':-)',
+                ':-('   : ':-(',
+                ';)'    : ';-)',
+                '|)'    : ':-|',
+                '|-)'   : ':-|',
+                '>:>'   : '^_^',
+                '<!>'   : ':!:',
+                '<:('   : ':-?',
+            }[text]
+        except KeyError:
+            return text