Compare commits

..

No commits in common. "9023627fe50f4222b214444d36531fa3e24fb12c" and "c3b5c6431c21d7bd03835c8dcafd18a8587a083d" have entirely different histories.

12 changed files with 408 additions and 852 deletions

4
.gitignore vendored
View file

@ -1,4 +0,0 @@
settings.local.cmd
*.log
*.pyc
out/

39
README Normal file
View file

@ -0,0 +1,39 @@
Complete MoinMoin to DokuWiki converter
Uses native MoinMoin modules to handle converting and translating paths.
Converts also page history and edit-log.
http://www.dokuwiki.org/tips:moinmoin2doku
Tested with MoinMoin 1.5 and DokuWiki 2012-09-10 releases
You need to run this on host where both MoinMoin and DokuWiki are configured,
it uses current configuration from both wikis.
Edit doku.php if your DokuWiki installation is other than /usr/share/dokuwiki
To convert moinmoin all pages with history, invoke:
$ ./moin2doku.py -a -d /var/lib/dokuwiki
To convert single page (FrontPage):
$ ./moin2doku.py -F moinmoin/data/pages/FrontPage -d out
after conversion be sure to fix ownership
(www-data:www-data being your uid/gid webserver runs):
# chown -R www-data:www-data /var/lib/dokuwiki/pages/*
# chown -R www-data:www-data /var/lib/dokuwiki/media/*
also, depending on your configuration, you may need to gzip the attic pages.
History:
version 0.1 2010-02
Slim Gaillard, based on the "extended python" convert.py script here:
https://www.dokuwiki.org/tips:moinmoin2doku?rev=1297006559#extended_python
version 0.2 2011
Elan Ruusamäe, moved to github, track history there
https://github.com/glensc/moin2doku
version 1.0 2012
Complete moinmoin to dokuwiki converter, uses native moinmoin code to handle
converting and translating paths. Converts also page history and edit-log.

View file

@ -1,93 +0,0 @@
Complete MoinMoin to DokuWiki converter
=======================================
Uses native MoinMoin modules to handle converting and translating paths.
Converts also page history and edit-log.
http://www.dokuwiki.org/tips:moinmoin2doku
Tested with MoinMoin 1.9.9 and DokuWiki 2018-04-22 releases under Windows 7
You need to run this on host where both MoinMoin and DokuWiki are configured,
it uses current configuration from both wikis.
Edit `doku.php` if your DokuWiki installation is other than `/usr/share/dokuwiki`
To convert moinmoin all pages with history, invoke:
```
$ ./moin2doku.py -a -d /var/lib/dokuwiki
```
To convert single page (FrontPage):
```
$ ./moin2doku.py -F moinmoin/data/pages/FrontPage -d out
```
You should invoke `bin/indexer.php` after conversion to make all pages are indexed.
and ensure ownership of files is correct:
(`www-data:www-data` being your uid/gid webserver runs):
```
# chown -R www-data:www-data /var/lib/dokuwiki/pages/*
# chown -R www-data:www-data /var/lib/dokuwiki/media/*
```
additionally, depending on your configuration, you may need to gzip the attic pages.
Hints for Windows Users
-----------------------
The Batchfiles (`*.cmd`) should help to do the conversation under Windows. You should
create a copy of the `settings.cmd` and call it `settings.local.cmd` to set your
own local paths.
Call `moin2doku.cmd` to convert the full MoinMoin Wiki. All DokuWiki pages will be
written to an `out` folder in the current directory.
This will convert a single page:
```
D:\moin2doku\> moin2doku.cmd MyMoinPage
```
Set `%OUTDIR%` to an alternativ output folder. This should not be the dokuwiki `data`
folder if you want to do a full conversation.
The `reindex.cmd` will call the `bin/indexer.php`-Skript.
History
=======
version 0.1 (2010-02)
-------------------
Slim Gaillard, based on the "extended python" convert.py script here:
https://www.dokuwiki.org/tips:moinmoin2doku?rev=1297006559#extended_python
version 0.2 (2011)
----------------
Elan Ruusamäe, moved to github, track history there
https://github.com/glensc/moin2doku
version 1.0 (2012)
----------------
Complete moinmoin to dokuwiki converter, uses native moinmoin code to handle
converting and translating paths. Converts also page history and edit-log.
This marks the project "done", I will no longer develop it or support it, as I got my conversion done. However, I do accept patches (pull requests) to sane amount.
I put repo online so others have better starting point than I did.
version 1.1 (2015)
----------------
Modifed the script to work with newer Moin versions and API changes.
version 1.2 (2019-01)
----------------
Some modifications to work with current DokuWiki and added more formattings.
Search GitHub Forks for newer versions of this project.

View file

@ -13,14 +13,10 @@
if ('cli' != php_sapi_name()) die(); if ('cli' != php_sapi_name()) die();
//add to following define of 'DOKU_INC' to your "doku.local.php" file and adjust the path: define('DOKU_INC', '/usr/share/dokuwiki/');
//define('DOKU_INC', '/home/caddy/wikifarm/dokuwiki/dokuwiki/');
//define('DOKU_INC', "d:/website/wwwroot/dokuwiki/" );
require_once './doku.local.php';
require_once DOKU_INC.'inc/init.php'; require_once DOKU_INC.'inc/init.php';
require_once DOKU_INC.'inc/common.php'; require_once DOKU_INC.'inc/common.php';
require_once DOKU_INC.'inc/cli.php'; require_once DOKU_INC.'inc/cliopts.php';
# disable gzip regardless of config, then we don't have to compress when converting # disable gzip regardless of config, then we don't have to compress when converting
$conf['compression'] = 0; //compress old revisions: (0: off) ('gz': gnuzip) ('bz2': bzip) $conf['compression'] = 0; //compress old revisions: (0: off) ('gz': gnuzip) ('bz2': bzip)
@ -33,33 +29,29 @@ function strip_dir($dir, $fn) {
return end(explode($dir.'/', $fn, 2)); return end(explode($dir.'/', $fn, 2));
} }
$action = $argv[1]; switch ($argv[1]) {
$argPage = $argv[2];
//filext = $argv[3];
switch ($action) {
case 'cleanID': case 'cleanID':
echo cleanID($argPage); echo cleanID($argv[2]);
break; break;
case 'wikiFN': case 'wikiFN':
if ($argc > 3 && $argv[3]) { if ($argc > 3 && $argv[3]) {
echo strip_dir($conf['olddir'], wikiFN($argPage, $argv[3])); echo strip_dir($conf['olddir'], wikiFN($argv[2], $argv[3]));
} else { } else {
echo strip_dir($conf['datadir'], wikiFN($argPage)); echo strip_dir($conf['datadir'], wikiFN($argv[2]));
} }
break; break;
case 'mediaFN': case 'mediaFN':
echo strip_dir($conf['mediadir'], mediaFN($argPage)); echo strip_dir($conf['mediadir'], mediaFN($argv[2]));
break; break;
case 'metaFN': case 'metaFN':
echo strip_dir($conf['metadir'], metaFN($argPage, $argv[3])); echo strip_dir($conf['metadir'], metaFN($argv[2], $argv[3]));
break; break;
case 'getNS': case 'getNS':
echo getNS($argPage); echo getNS($argv[2]);
break; break;
case 'getId': case 'getId':
echo getId(); echo getId();
break; break;
default: default:
die("Unknown knob: {$action}"); die("Unknown knob: {$argv[1]}");
} }

18
doku.py Executable file → Normal file
View file

@ -11,9 +11,6 @@
import sys import sys
import subprocess import subprocess
from MoinMoin import log
logging = log.getLogger(__name__)
class DokuWiki: class DokuWiki:
def __init__(self): def __init__(self):
self.callcache = {} self.callcache = {}
@ -27,17 +24,10 @@ class DokuWiki:
def __call(self, method, *args): def __call(self, method, *args):
args = list(args) args = list(args)
uargs = [] key = "%s:%s" % (method, ",".join(args))
for arg in args:
try:
arg.decode('utf-8')
#already UTF-8 ready
uargs.append(arg)
except UnicodeError:
uargs.append(arg.encode('utf-8'))
key = "%s:%s" % (method, ",".join(uargs))
if not self.callcache.has_key(key): if not self.callcache.has_key(key):
cmd = ['php', './doku.php', method] + [arg.encode("utf-8") for arg in uargs] cmd = ['./doku.php', method ] + args
res = subprocess.Popen(cmd, stdin = None, stdout = subprocess.PIPE, stderr = sys.stderr, close_fds = False).communicate() res = subprocess.Popen(cmd, stdin = None, stdout = subprocess.PIPE, stderr = sys.stderr, close_fds = True).communicate()
self.callcache[key] = unicode(res[0].decode('utf-8')) self.callcache[key] = unicode(res[0].decode('utf-8'))
print "%s->%s" % (cmd, self.callcache[key])
return self.callcache[key] return self.callcache[key]

View file

@ -1,36 +0,0 @@
@echo off
setlocal
call settings.cmd
if not "%1"=="" goto :singlePage %1
if "%OUTDIR%"=="" (
call :deldir out\attic || (pause & goto :eof)
call :deldir out\media || (pause & goto :eof)
call :deldir out\meta || (pause & goto :eof)
call :deldir out\pages || (pause & goto :eof)
if exist %~n0.pages.log del %~n0.pages.log
if not exist out md out
set OUTDIR=%CD%\out
)
call python moin2doku.py %DOKU_FULL_HISTORY% -d "%OUTDIR:\=/%" >"%~n0.log" 2>"%~n0.err.log"
goto :eof
:deldir
if exist %1 rd /s/q %1
if exist %1 exit /B 1
goto :eof
:singlePage
if "%OUTDIR%"=="" set OUTDIR=%CD%\out
call python moin2doku.py %DOKU_FULL_HISTORY% -p "%MOIN_DATA_HOME%\pages\%~1" -f -d "%OUTDIR:\=/%" >>"%~n0.log" 2>>"%~n0.err.log" || type "%~n0.err.log"
if %ERRORLEVEL% == 0 if exist "%DOKU_ANIMALS_HOME%\%ANIMAL%\conf\local.php" (
rem touching "%DOKU_ANIMALS_HOME%\%ANIMAL%\conf\local.php" to invalidate cache
pushd "%DOKU_ANIMALS_HOME%\%ANIMAL%\conf"
copy /y/b local.php +,, >nul
popd
)
goto :eof

View file

@ -11,7 +11,7 @@
import sys, os, os.path, re, codecs import sys, os, os.path, re, codecs
import getopt import getopt
from MoinMoin import user, wikiutil from MoinMoin import user, wikiutil
from MoinMoin.web.contexts import ScriptContext as RequestCLI from MoinMoin.request import RequestCLI
from MoinMoin.logfile import editlog from MoinMoin.logfile import editlog
from MoinMoin.Page import Page from MoinMoin.Page import Page
from shutil import copyfile, copystat from shutil import copyfile, copystat
@ -19,11 +19,6 @@ from os import listdir, mkdir
from os.path import isdir, basename from os.path import isdir, basename
from doku import DokuWiki from doku import DokuWiki
from moinformat import moin2doku from moinformat import moin2doku
import random
# sys.setdefaultencoding() does not exist, here!
reload(sys) # Reload does the trick!
sys.setdefaultencoding('cp1252')
USEC = 1000000 USEC = 1000000
@ -45,8 +40,8 @@ def init_dirs(output_dir):
mkdir(metadir) mkdir(metadir)
def readfile(filename): def readfile(filename):
f = open(filename, 'r') with open(filename, 'r') as f:
text = f.read() text = f.read()
return unicode(text.decode('utf-8')) return unicode(text.decode('utf-8'))
def writefile(filename, content, overwrite=False): def writefile(filename, content, overwrite=False):
@ -55,7 +50,7 @@ def writefile(filename, content, overwrite=False):
os.makedirs(dir); os.makedirs(dir);
if os.path.exists(filename) and overwrite == False: if os.path.exists(filename) and overwrite == False:
raise (OSError, 'File already exists: %s' % filename) raise OSError, 'File already exists: %s' % filename
# ensure it's a list # ensure it's a list
if not isinstance(content, (list, tuple)): if not isinstance(content, (list, tuple)):
@ -65,14 +60,9 @@ def writefile(filename, content, overwrite=False):
f.writelines([line + u'\n' for line in content]) f.writelines([line + u'\n' for line in content])
f.close() f.close()
def encode_relaxed(text):
return text.encode("ascii", errors="ignore")
# page = MoinMoin Page oject # page = MoinMoin Page oject
# ns = DokuWiki namespace where attachments to copy # ns = DokuWiki namespace where attachments to copy
def copy_attachments(page, ns,randomID): def copy_attachments(page, ns):
srcdir = page.getPagePath('attachments', check_create = 0) srcdir = page.getPagePath('attachments', check_create = 0)
if not isdir(srcdir): if not isdir(srcdir):
return return
@ -83,13 +73,10 @@ def copy_attachments(page, ns,randomID):
attachments = listdir(srcdir) attachments = listdir(srcdir)
for attachment in attachments: for attachment in attachments:
try: src = os.path.join(srcdir, attachment)
src = os.path.join(srcdir, attachment) dst = os.path.join(output_dir, 'media', dw.mediaFN(dw.cleanID("%s/%s" % (ns, attachment))))
dst = os.path.join(output_dir, 'media', dw.mediaFN(dw.cleanID(u"%s/%s" % (ns, str(randomID)+attachment)))) copyfile(src, dst)
copyfile(src, dst) copystat(src, dst)
copystat(src, dst)
except UnicodeDecodeError:
print 'ERROR: unable to convert attachment "%s"' % encode_relaxed(attachment)
def print_help(): def print_help():
program = sys.argv[0] program = sys.argv[0]
@ -169,16 +156,13 @@ def convert_editlog(page, output = None, overwrite = False):
def convertfile(page, output = None, overwrite = False): def convertfile(page, output = None, overwrite = False):
pagedir = page.getPagePath() pagedir = page.getPagePath()
pagename = wikiname(pagedir) pagename = wikiname(pagedir)
if not output: if not output:
output = pagename output = pagename
print "Converting %s" % encode_relaxed(pagename)
if page.isUnderlayPage(): if page.isUnderlayPage():
print "underlay: %s" % page.request.cfg.data_underlay_dir print "underlay: %s" % page.request.cfg.data_underlay_dir
print "underlay: %s" % request.cfg.data_underlay_dir print "underlay: %s" % request.cfg.data_underlay_dir
print "SKIP UNDERLAY: %s" % encode_relaxed(pagename) print "SKIP UNDERLAY: %s" % pagename
return False return False
current_exists = page.exists() current_exists = page.exists()
@ -189,9 +173,6 @@ def convertfile(page, output = None, overwrite = False):
else: else:
revs = [current_rev] revs = [current_rev]
# Generate random ID Number for collision avoidance when attachments in Namespace have the same name
randomID = random.randint(101,999)
for rev in revs: for rev in revs:
page = Page(request, pagename, rev = rev) page = Page(request, pagename, rev = rev)
pagefile, realrev, exists = page.get_rev(rev = rev); pagefile, realrev, exists = page.get_rev(rev = rev);
@ -207,7 +188,7 @@ def convertfile(page, output = None, overwrite = False):
print "recovered %s: %s" % (rev, mtime) print "recovered %s: %s" % (rev, mtime)
if not mtime: if not mtime:
print "NO REVISION: for %s" % encode_relaxed(pagefile) print "NO REVISION: for %s" % pagefile
continue continue
if rev == current_rev: if rev == current_rev:
@ -218,7 +199,7 @@ def convertfile(page, output = None, overwrite = False):
else: else:
out_file = os.path.join(output_dir, 'attic', dw.wikiFN(output, str(mtime))) out_file = os.path.join(output_dir, 'attic', dw.wikiFN(output, str(mtime)))
content = moin2doku(pagename, page.get_raw_body(),randomID) content = moin2doku(pagename, page.get_raw_body())
if len(content) == 0: if len(content) == 0:
# raise Exception, "No content" # raise Exception, "No content"
print "NO CONTENT: exists: %s,%s" % (exists, os.path.exists(pagefile)) print "NO CONTENT: exists: %s,%s" % (exists, os.path.exists(pagefile))
@ -227,7 +208,7 @@ def convertfile(page, output = None, overwrite = False):
copystat(pagefile, out_file) copystat(pagefile, out_file)
ID = dw.cleanID(output) ID = dw.cleanID(output)
copy_attachments(page, dw.getNS(ID),randomID) copy_attachments(page, dw.getNS(ID))
# convert edit-log, it's always present even if current page is not # convert edit-log, it's always present even if current page is not
convert_editlog(page, output = output, overwrite = overwrite) convert_editlog(page, output = output, overwrite = overwrite)
@ -242,7 +223,7 @@ def convertfile(page, output = None, overwrite = False):
if old_page != ID: if old_page != ID:
redirect_map[old_page] = ID redirect_map[old_page] = ID
print "Converted %s as %s" % (encode_relaxed(pagename), dw.wikiFN(output)) print "Converted %s as %s" % (pagename, dw.wikiFN(output))
return True return True
@ -313,7 +294,6 @@ else:
# get list of all pages in wiki # get list of all pages in wiki
# hide underlay dir temporarily # hide underlay dir temporarily
underlay_dir = request.rootpage.cfg.data_underlay_dir underlay_dir = request.rootpage.cfg.data_underlay_dir
print(underlay_dir)
request.rootpage.cfg.data_underlay_dir = None request.rootpage.cfg.data_underlay_dir = None
pages = request.rootpage.getPageList(user = '', exists = not convert_attic, filter = filter) pages = request.rootpage.getPageList(user = '', exists = not convert_attic, filter = filter)
pages = dict(zip(pages, pages)) pages = dict(zip(pages, pages))
@ -327,11 +307,6 @@ else:
del pages[frontpage.page_name] del pages[frontpage.page_name]
pages[dw.getId()] = frontpage.page_name pages[dw.getId()] = frontpage.page_name
print "--------------------------------------------------"
for output, pagename in pages.items():
print " - %s" % encode_relaxed(pagename)
print "--------------------------------------------------"
converted = 0 converted = 0
for output, pagename in pages.items(): for output, pagename in pages.items():
page = Page(request, pagename) page = Page(request, pagename)

View file

@ -7,22 +7,20 @@
# #
# Author: Elan Ruusamäe <glen@pld-linux.org> # Author: Elan Ruusamäe <glen@pld-linux.org>
# Version: 1.0 # Version: 1.0
#from MoinMoin.web.request import Request as RequestCLI
from MoinMoin.web.contexts import ScriptContext from MoinMoin import wikimacro, wikiutil
from MoinMoin import wikiutil
from MoinMoin.Page import Page from MoinMoin.Page import Page
from MoinMoin.parser.text_moin_wiki import Parser from MoinMoin.parser.wiki import Parser
from text_dokuwiki import Formatter from text_dokuwiki import Formatter
from MoinMoin.request import RequestCLI
import sys import sys
import StringIO import StringIO
def moin2doku(pagename, text, randomID=None): def moin2doku(pagename, text):
parser = Parser(text, request) parser = Parser(text, request)
formatter.setRandomID(randomID)
# this needed for macros # this needed for macros
request.formatter = formatter request.formatter = formatter
@ -39,7 +37,7 @@ def moin2doku(pagename, text, randomID=None):
return unicode(output.getvalue().decode('utf-8')) return unicode(output.getvalue().decode('utf-8'))
request = ScriptContext() request = RequestCLI()
formatter = Formatter(request) formatter = Formatter(request)
if __name__ == "__main__": if __name__ == "__main__":
@ -49,6 +47,6 @@ if __name__ == "__main__":
else: else:
inputfile = 'syntaxreference.txt' inputfile = 'syntaxreference.txt'
f = open(inputfile, 'r') with open(inputfile, 'r') as f:
text = f.read() text = f.read()
print moin2doku('test', text) print moin2doku('test', text)

View file

@ -1,69 +0,0 @@
@echo off
setlocal
call settings.cmd
pushd "%DOKU_ANIMALS_HOME%\%ANIMAL%"
if not exist data\pages\ (
echo WARNUNG:
echo Es wurden kein "pages" Verzeichnis gefunden. Wurde die neue Version bereits
echo ins Zielverzeichnis kopiert?
pause
goto :eof
)
if not exist data\media\logo.png (
echo working in %CD%
rem call :cleanup data\attic || goto :ende
call :cleanup data\cache || goto :ende
call :cleanup data\index || goto :ende
call :cleanup data\locks || goto :ende
call :cleanup data\media_attic || goto :ende
call :cleanup data\media_meta || goto :ende
call :cleanup data\tmp || goto :ende
xcopy /S/I/Y/Q common\*.* data
)
REM if exist data\pages\startseiteneu.txt (
REM ren data\pages\startseiteneu.txt startseite.txt
REM ren data\meta\startseiteneu.changes startseite.changes
REM )
popd
pushd "%DOKU_HOME%"
php bin\indexer.php || goto :ende
popd
pushd "%DOKU_ANIMALS_HOME%\%ANIMAL%"
if exist data\meta\_dokuwiki.changes del data\meta\_dokuwiki.changes
if exist data\meta\_dokuwiki.changes del data\meta\_dokuwiki.changes
(
for /F "delims=*" %%D in ('dir /b/S/A:D data\meta\*.*') do (
if exist "%%D\*.changes" type "%%D\*.changes" 2>nul
)
) > _dokuwiki_unsorted.changes
sort _dokuwiki_unsorted.changes /O data\meta\_dokuwiki.changes
del _dokuwiki_unsorted.changes
echo --- compressing old files
for /F "delims=*" %%T in ('dir data\attic\*.txt /s/b') do (
"c:\Program Files\7-Zip\7z.exe" a -bso0 "%%T.gz" "%%T" && del "%%T" || goto :ende
)
echo --- done
:ende
popd
pause
goto :eof
:cleanup
if not exist %1 goto :eof
echo cleaning up %1
rd /s/q %1
if exist %1 exit /b 1
md %1
echo >nul 2>"%~1\_dummy"

View file

@ -1,45 +0,0 @@
@echo off
REM -- no setlocal in this script!
REM make a copy of this file and adjust the paths
set PHP_HOME=c:\Program Files\php
set PYTHON_HOME=c:\Python27
REM -- MoinMoin settings
set MOIN_HOME=c:\wwwroot\wiki\moin
set MOIN_CONFIG=%MOIN_HOME%\wiki\config
REM set MOIN_CONFIG=c:\wwwroot\moinfarmdata\config
set MOIN_DATA_HOME=%MOIN_HOME%\wiki\data
REM set MOIN_CONFIG=c:\wwwroot\moinfarmdata\<farmwiki>
REM -- DokuWiki settings
set DOKU_HOME=c:\wwwroot\wiki\dokuwiki
set DOKU_ANIMALS_HOME=%DOKU_HOME%
REM set DOKU_ANIMALS_HOME=c:\wwwroot\dokufarmdata
REM set animal=<yourFarmAnimalName>
REM comment this in to do a full converstion
REM set DOKU_FULL_HISTORY=-a
REM -- path to your php.ini used by your webserver
REM set PHP_INI_SCAN_DIR=c:\Program Files\ApacheHttpd\conf\
REM -- set this to your "production" dokuwiki if you want to update only.
REM set OUTDIR=%DOKU_ANIMALS_HOME%\%animal%\data
REM ----8<--------8<--------8<--------8<--------8<--------8<--------8<----
REM -- remove everything beyond the line from your copy
chcp 1252
if exist "%~dpn0.local.cmd" call "%~dpn0.local.cmd"
REM -- %animal% must be lowercase!
(
set animal=
set animal=%animal%
)
set PATH=%PATH%;%PYTHON_HOME%;%PHP_HOME%
set PYTHONPATH=.
set PYTHONPATH=%PYTHONPATH%;%MOIN_HOME%
set PYTHONPATH=%PYTHONPATH%;%MOIN_HOME%\MoinMoin\support
set PYTHONPATH=%PYTHONPATH%;%MOIN_CONFIG%
set PYTHONPATH=%PYTHONPATH%;

0
syntaxreference.txt Executable file → Normal file
View file

855
text_dokuwiki.py Executable file → Normal file
View file

@ -1,537 +1,346 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
# Setup VIM: ex: noet ts=2 sw=2 : # Setup VIM: ex: noet ts=2 sw=2 :
""" """
MoinMoin - Dokuwiki Formatter MoinMoin - Dokuwiki Formatter
@copyright: 2000, 2001, 2002 by Jürgen Hermann <jh@web.de> @copyright: 2000, 2001, 2002 by Jürgen Hermann <jh@web.de>
@copyright: 2011-2012 Elan Ruusamäe <glen@delfi.ee> @copyright: 2011-2012 Elan Ruusamäe <glen@delfi.ee>
@license: GNU GPL, see COPYING for details. @license: GNU GPL, see COPYING for details.
""" """
from xml.sax import saxutils from xml.sax import saxutils
from MoinMoin.formatter import FormatterBase from MoinMoin.formatter.base import FormatterBase
from MoinMoin import config from MoinMoin import config
from MoinMoin.Page import Page from MoinMoin.Page import Page
from types import * from types import *
from MoinMoin import log
logging = log.getLogger(__name__)
# TODO: let base class MoinMoin/formatter/base.py handle not implemented methods # TODO: let base class MoinMoin/formatter/base.py handle not implemented methods
class Formatter(FormatterBase): class Formatter(FormatterBase):
""" """
Send Dokuwiki formatted data. Send Dokuwiki formatted data.
""" """
hardspace = '&nbsp;' hardspace = '&nbsp;'
# hardspace = ' ' # hardspace = ' '
def __init__(self, request, **kw): def __init__(self, request, **kw):
apply(FormatterBase.__init__, (self, request), kw) apply(FormatterBase.__init__, (self, request), kw)
self._current_depth = 1 self._current_depth = 1
self._base_depth = 0 self._base_depth = 0
self.in_pre = 0 self.in_pre = 0
self.in_table = 0 self.in_table = 0
self._text = None # XXX does not work with links in headings!!!!! self._text = None # XXX does not work with links in headings!!!!!
self.randomID= None
self.list_depth = 0 self.list_depth = 0
self.list_type = ' ' self.list_type = ' '
def setRandomID(self,ID): def _escape(self, text, extra_mapping={"'": "&apos;", '"': "&quot;"}):
self.randomID = str(ID) return saxutils.escape(text, extra_mapping)
def _escape(self, text, extra_mapping={"'": "&apos;", '"': "&quot;"}): def startDocument(self, pagename):
return saxutils.escape(text, extra_mapping) encoding = config.charset
return '<?xml version="1.0" encoding="%s"?>\n<s1 title="%s">' % (
def startDocument(self, pagename): encoding, self._escape(pagename))
encoding = config.charset
return '<?xml version="1.0" encoding="%s"?>\n<s1 title="%s">' % ( def endDocument(self):
encoding, self._escape(pagename)) result = ""
while self._current_depth > 1:
def endDocument(self): result += "</s%d>" % self._current_depth
result = "" self._current_depth -= 1
while self._current_depth > 1: return result + '</s1>'
result += "</s%d>" % self._current_depth
self._current_depth -= 1 def lang(self, on, lang_name):
return result + '</s1>' return ('<div lang="">' % lang_name, '</div>')[not on]
def lang(self, on, lang_name): def sysmsg(self, on, **kw):
return ('<div lang="">' % lang_name, '</div>')[not on] return ('<sysmsg>', '</sysmsg>')[not on]
def sysmsg(self, on, **kw): def rawHTML(self, markup):
return ('<sysmsg>', '</sysmsg>')[not on] return '<html>' + markup + '</html>'
def rawHTML(self, markup): def pagelink(self, on, pagename='', page=None, **kw):
#return '<html>' + markup + '</html>' if on:
return '' return '[[:' + ":".join(pagename.split("/")) + "|"
else:
def pagelink(self, on, pagename='', page=None, **kw): return ']]'
if on:
return '[[:' + ":".join(pagename.split("/")) + "|" def interwikilink(self, on, interwiki='', pagename='', **kw):
else: if on:
return ']]' if interwiki == 'Self':
return self.pagelink(on, pagename, **kw)
def interwikilink(self, on, interwiki='', pagename='', **kw): return '[[%s>%s|' % (interwiki, pagename)
if on: else:
if interwiki == 'Self': return ']]'
return self.pagelink(on, pagename, **kw)
interwikis = { def url(self, on, url='', css=None, **kw):
'WikiPedia':'wp', return ('[[%s|' % (self._escape(url)), ']]')[not on]
'FrWikiPedia':'wpfr',
'DeWikiPedia':'wpde', def attachment_link(self, url, text, **kw):
'MetaWikiPedia':'wpmeta' return '{{%s|%s}}' % (url, text)
}
if interwiki in interwikis: def attachment_image(self, url, **kw):
return '[[%s>%s|' % (interwikis.get(interwiki), pagename) return '{{%s|}}' % (url,)
return '[[%s>%s|' % (interwiki, pagename)
else: def attachment_drawing(self, url, text, **kw):
return ']]' return '{{%s|%s}}' % (url, text)
def url(self, on, url='', css=None, **kw): def text(self, text, **kw):
return ('[[%s|' % (self._escape(url)), ']]')[not on] self._did_para = 0
if self._text is not None:
def attachment_link(self, on, url=None, querystr=None, **kw): self._text.append(text)
if on: return text
return '{{ %s | ' % (self.randomID+url)
else: def rule(self, size=0, **kw):
return ' }}' # size not supported
if size >= 4:
def attachment_image(self, url, **kw): return '----\n'
return '{{%s|}}' % (self.randomID+url,) else:
return '-' * size + '\n'
def attachment_drawing(self, url, text, **kw):
return '{{%s|%s}}' % (self.randomID+url, text) def icon(self, type):
return '<icon type="%s" />' % type
def text(self, text, **kw):
self._did_para = 0 def strong(self, on, **kw):
if self._text is not None: return ['**', '**'][not on]
self._text.append(text)
return text def emphasis(self, on, **kw):
return ['//', '//'][not on]
def rule(self, size=0, **kw):
# size not supported def highlight(self, on, **kw):
if size >= 4: return ['**', '**'][not on]
return '----\n'
else: def number_list(self, on, type=None, start=None, **kw):
return '-' * size + '\n' # list type not supported
if on:
def icon(self, type): self.list_depth += 1
return '<icon type="%s" />' % type self.list_type = '-'
else:
def strong(self, on, **kw): self.list_depth -= 1
return ['**', '**'][not on] self.list_type = ' '
def emphasis(self, on, **kw): return ['', '\n'][on]
return ['//', '//'][not on]
def bullet_list(self, on, **kw):
def highlight(self, on, **kw): if on:
return ['**', '**'][not on] self.list_depth += 1
self.list_type = '*'
def number_list(self, on, type=None, start=None, **kw): else:
# list type not supported self.list_depth -= 1
if on: self.list_type = ' '
self.list_depth += 1
self.list_type = '-' return ['', '\n'][on]
else:
self.list_depth -= 1 def listitem(self, on, **kw):
self.list_type = ' ' # somewhy blockquote uses "listitem" call
return [(' ' * self.list_depth * 2) + self.list_type + ' ', '\n'][not on]
return ['', '\n'][on]
def code(self, on, **kw):
def bullet_list(self, on, **kw): """ `typewriter` or {{{typerwriter}}, for code blocks i hope codeblock works """
if on: return ["''", "''"][not on]
self.list_depth += 1
self.list_type = '*' def sup(self, on, **kw):
else: return ['<sup>', '</sup>'][not on]
self.list_depth -= 1
if self.list_depth <= 0: def sub(self, on, **kw):
self.list_type = ' ' return ['<sub>', '</sub>'][not on]
return ['', '\n'][on] def strike(self, on, **kw):
return ['<del>', '</del>'][not on]
# generic transclude/include:
def transclusion(self, on, **kw): def preformatted(self, on, **kw):
return '' FormatterBase.preformatted(self, on)
result = ''
def transclusion_param(self, **kw): if self.in_p:
return '' result = self.paragraph(0)
return result + ['<file>', '</file>\n'][not on]
def listitem(self, on, **kw):
# somewhy blockquote uses "listitem" call def paragraph(self, on, **kw):
return [(' ' * self.list_depth * 2) + self.list_type + ' ', '\n'][not on] FormatterBase.paragraph(self, on)
if self.in_table or self.list_depth:
def code(self, on, **kw): return ''
""" `typewriter` or {{{typerwriter}}, for code blocks i hope codeblock works """ return ['', '\n\n'][not on]
return ["''%%", "%%''"][not on]
def linebreak(self, preformatted=1):
def sup(self, on, **kw): return ['\n', '\\\n'][not preformatted]
return ['<sup>', '</sup>'][not on]
def heading(self, on, depth, **kw):
def sub(self, on, **kw): # heading depth reversed in dokuwiki
return ['<sub>', '</sub>'][not on] heading_depth = 7 - depth
def strike(self, on, **kw): if on:
return ['<del>', '</del>'][not on] return u'%s ' % (u'=' * heading_depth)
else:
def small(self, on, **kw): return u' %s\n' % (u'=' * heading_depth)
#https://www.dokuwiki.org/plugin:wrap
return ['<wrap lo>', '</wrap>'][not on] def table(self, on, attrs={}, **kw):
if on:
def big(self, on, **kw): self.in_table = 1
#https://www.dokuwiki.org/plugin:wrap else:
return ['<wrap hi>', '</wrap>'][not on] self.in_table = 0
return ''
def preformatted(self, on, **kw):
FormatterBase.preformatted(self, on) def table_row(self, on, attrs={}, **kw):
result = '' return ['\n', '|'][not on]
if self.in_p:
result = self.paragraph(0) def table_cell(self, on, attrs={}, **kw):
return result + ['<code>', '</code>\n'][not on] return ['|', ''][not on]
def paragraph(self, on, **kw): def anchordef(self, id):
FormatterBase.paragraph(self, on) # not supported
if self.in_table or self.list_depth: return ''
return ''
return ['', '\n\n'][not on] def anchorlink(self, on, name='', **kw):
# kw.id not supported, we hope the anchor matches existing heading on page
def linebreak(self, preformatted=1): return ('[[#', ']]') [not on]
return ['\n', '\\\n'][not preformatted]
def underline(self, on, **kw):
def heading(self, on, depth, **kw): return ['__', '__'][not on]
# heading depth reversed in dokuwiki
heading_depth = 7 - depth def definition_list(self, on, **kw):
result = ''
if on: if self.in_p:
return u'%s ' % (u'=' * heading_depth) result = self.paragraph(0)
else: return result + ['<gloss>', '</gloss>'][not on]
return u' %s\n' % (u'=' * heading_depth)
def definition_term(self, on, compact=0, **kw):
def table(self, on, attrs={}, **kw): return ['<label>', '</label>'][not on]
if on:
self.in_table = 1 def definition_desc(self, on, **kw):
else: return ['<item>', '</item>'][not on]
self.in_table = 0
return ['', '\n'][not on] def image(self, src=None, **kw):
valid_attrs = ['src', 'width', 'height', 'alt', 'title']
def table_row(self, on, attrs={}, **kw):
return ['\n', '|'][not on] url = src
if '?' in url:
def table_cell(self, on, attrs={}, **kw): url += '&'
return ['|', ''][not on] else:
url += '?'
def anchordef(self, id):
# https://www.dokuwiki.org/plugin:anchor attrs = {}
return '{{anchor:'+id+'}}' for key, value in kw.items():
if key in valid_attrs:
def anchorlink(self, on, name='', **kw): attrs[key] = value
# kw.id not supported, we hope the anchor matches existing heading on page
return ('[[#', ']]') [not on] # TODO: finish this
if attrs.has_key('width'):
def underline(self, on, **kw): url += attrs['width']
return ['__', '__'][not on]
return '{{' + url + '}}'
def definition_list(self, on, **kw):
# https://www.dokuwiki.org/plugin:definitionlist def code_area(self, on, code_id, code_type='code', show=0, start=-1, step=-1):
result = '' syntax = ''
if self.in_p: # switch for Python: http://simonwillison.net/2004/may/7/switch/
result = self.paragraph(0) try:
return result syntax = {
'ColorizedPython': 'python',
def definition_term(self, on, compact=0, **kw): 'ColorizedPascal': 'pascal',
#MoinMoin does no wiki markup in DL-Terms 'ColorizedJava': 'java',
return [' ;%%', '%%\n'][not on] 'ColorizedCPlusPlus': 'cpp',
}[code_type]
def definition_desc(self, on, **kw): except KeyError:
return [' :', '\n'][not on] pass
def image(self, src=None, **kw): return ('<code %s>' % syntax , '</code>')[not on]
valid_attrs = ['src', 'width', 'height', 'alt', 'title']
def code_line(self, on):
url = src return ('', '\n')[on]
if '?' in url:
url += '&' def code_token(self, on, tok_type):
else: # not supported
url += '?' return ''
attrs = {} def comment(self, text):
for key, value in kw.items(): # real comments (lines with two hash marks)
if key in valid_attrs: if text[0:2] == '##':
attrs[key] = value return "/* %s */\n" % text[2:].strip()
# TODO: finish this # Some kind of Processing Instruction
if attrs.has_key('width'): # http://moinmo.in/HelpOnProcessingInstructions
url += attrs['width'] tokens = text.lstrip('#').split(None, 1)
if tokens[0] in ('language', 'format', 'refresh'):
return '{{' + url + '}}' return ''
def code_area(self, on, code_id, code_type='code', show=0, start=-1, step=-1,msg=None): if tokens[0] == 'acl':
syntax = '' # TODO: fill acl.auth.php
# switch for Python: http://simonwillison.net/2004/may/7/switch/ return ''
try:
syntax = { if tokens[0] == 'deprecated':
'ColorizedPython': 'python', return '<note warning>This page is deprecated</note>\n'
'ColorizedPascal': 'pascal',
'ColorizedJava': 'java', if tokens[0] == 'redirect':
'ColorizedCPlusPlus': 'cpp', return text + "\n"
}[code_type]
except KeyError: if tokens[0] == 'pragma':
pass # TODO: can do 'description' via 'meta' dokuwiki plugin
return "/* pragma: %s */\n" % " ".join(tokens[1:])
return ('<code %s>' % syntax , '</code>')[not on]
return "/* %s */\n" % text.lstrip('#')
def code_line(self, on):
return ('', '\n')[on] def macro(self, macro_obj, name, args):
def email(args):
def code_token(self, on, tok_type): mail = args.replace(' AT ', '@')
# not supported mail = mail.replace(' DOT ', '.')
return '' return '[[%s|%s]]' % (mail, args)
def comment(self, text): # function which will just do what parent class would
# real comments (lines with two hash marks) def inherit(args):
if text[0:2] == '##': return apply(FormatterBase.macro, (self, macro_obj, name, args))
# https://www.dokuwiki.org/plugin:comment
comment = text[2:].strip() try:
if len(comment)>1: lookup = {
return "/* %s */\n" % text[2:].strip() 'BR' : '\\\\',
return '\n' 'MailTo' : email,
'GetText' : args,
# Some kind of Processing Instruction 'ShowSmileys' : inherit,
# http://moinmo.in/HelpOnProcessingInstructions }[name]
tokens = text.lstrip('#').split(None, 1) except KeyError:
if tokens[0] in ('language', 'format', 'refresh'): lookup = '/* UndefinedMacro: %s(%s) */' % (name, args)
return ''
if type(lookup) == FunctionType:
if tokens[0] == 'acl': text = lookup(args)
# TODO: fill acl.auth.php else:
logging.info('SKIPPING ACL: %s', text) text = lookup
return '' return text
if tokens[0] == 'deprecated': def smiley(self, text):
return '<note warning>This page is deprecated</note>\n' try:
# https://www.dokuwiki.org/devel:smileys.conf
if tokens[0] == 'redirect': return {
return text + "\n" # note: reverse sorted so that longer smileys get matched first
'X-(' : ':-X',
if tokens[0] == 'pragma': '{X}' : ':!:',
# TODO: can do 'description' via 'meta' dokuwiki plugin '{*}' : '<ubu>',
pargs = tokens[1].split(None, 1) '(./)' : u'',
if pargs[0]=='section-numbers': ':))' : ':-P',
return '/* meta: %s */' % tokens ':-))' : ':-P',
logging.info('SKIPPING PRAGMA: %s', tokens) ':-?' : ':-P',
#return "/* pragma: %s */\n" % " ".join(tokens[1:]) ':o' : ':-o',
return '' '{OK}' : ':!:',
'{o}' : '<circ>',
#return "/* %s */\n" % text.lstrip('#') '{i}' : ':!:',
return '' ':D' : ':-D',
'B)' : '8-)',
'B-)' : '8-)',
def macro(self, macro_obj, name, args,markup): '{3}' : '<3>',
def email(args): '{2}' : '<2>',
mail = args.replace(' AT ', '@') '{1}' : '<1>',
mail = mail.replace(' DOT ', '.') '(!)' : ':!:',
return '[[%s|%s]]' % (mail, args) '/!\\' : ':!:',
':\\' : ':-\\',
def showAttachedFiles(args): ':))' : ':-)',
args = args.split(',') ':)' : ':-)',
if len(args)>1: ':(' : ':-(',
return '{{ %s | %s }}' % (self.randomID+args[0].strip(), args[1].strip()) ':-))' : ':-)',
else: ':-)' : ':-)',
return '' ':-(' : ':-(',
';)' : ';-)',
# function which will just do what parent class would '|)' : ':-|',
def inherit(args): '|-)' : ':-|',
return apply(FormatterBase.macro, (self, macro_obj, name, args)) '>:>' : '^_^',
'<!>' : ':!:',
def randomQuote(args): '<:(' : ':-?',
# https://www.dokuwiki.org/plugin:xfortune }[text]
return '{{xfortune>quote:'+args+'.txt}}' except KeyError:
return text
def monthcal(args):
# https://www.dokuwiki.org/plugin:monthcal
selfname = self.page.page_name
return '{{monthcal:create_links=short,namespace='+selfname.replace('/',':')+'}}'
def navigation(args):
# https://www.dokuwiki.org/plugin:alphaindex
selfname = self.page.page_name
args = args.split(',')
if len(args)>0:
try:
result = {
'slides': '[<>]',
'children': '{{alphaindex>:%s#1|nons incol}}' % selfname.replace('/',':'),
'siblings': '{{alphaindex>.#1|nons incol}}',
'slideshow': '/* no support for slideshow navigation */'
}[args[0].strip()]
except KeyError:
result = '/* Unknown Navigation: %s #%s#*/' % args, args[0].strip()
else:
result = '/* Unsupported Navigation: %s */' % args
return result
def footnote(args):
return '((%s))' % args
def dateTimeMacro(args):
#https://www.dokuwiki.org/plugin:date
#args = args.split(',');
return '{{date>%%c|timestamp=strtotime("%s")|locale=de}}' % args
def dateMacro(args):
#https://www.dokuwiki.org/plugin:date
#args = args.split(',');
return '{{date>%%x|timestamp=strtotime("%s")|locale=de}}' % args
def includeMacro(args):
#https://www.dokuwiki.org/plugin:include
#logging.info('Include(%s)' % args)
args = map(unicode.strip, args.split(','));
#dokupage = ":".join(pagename.split("/"))
if len(args)==1:
return '{{page>%s&nodate}}' % ":".join(args[0].split("/"))
elif(u'titlesonly' in args):
#https://www.dokuwiki.org/plugin:changes
#https://www.dokuwiki.org/plugin:pagelist
selfname = self.page.page_name
selfNs = ":".join(selfname.split("/")).lower()
pairs = [arg.split('=') for arg in args]
# attrs = {}
# for key, value in pairs:
# attrs[key] = value
#logging.info('pairs:"%s"' % pairs)
incName = ''#pairs[0]
incCount = -1
incTitlesOnly = False
notNamedParam = 0
for pair in pairs:
if len(pair)==1:
if u'titlesonly'==pair:
notNamedParam = -1
incTitlesOnly = True
elif notNamedParam >=0:
if notNamedParam==0:
incName = pairs[notNamedParam]
notNamedParam += 1;
else:
notNamedParam = -1
if u'items'==pair[0]:
incCount = int(pair[1])
resultArgs = '-h1 -textPages=""'
#(keys,values) = map()
if incCount > 0:
resultArgs += ' -idAndTitle -simpleList -sortId -nbItemsMax=%d' % incCount
else:
resultArgs += ' -nbCol=2'
##<nspages fortran:mailarchiv -pregPagesOn="/2.*/" -h1 -nbCol=2 -textPages="">
## Lister der letzten 10 Mails:
##<nspages .:mailarchiv -nbItemsMax=10 -sortId -simpleList -idAndTitle -reverse -h1 -nbCol=1 -textPages="">
if incName[0]=='^':
nspagedelim = incName.rfind('/')
ns = ":".join(incName[1:nspagedelim].split('/')).lower()
incPageReg = incName[(nspagedelim+1):]
resultArgs += ' -pregPagesOn="/^%s/"' % incPageReg
else:
ns = selfNs
return '<nspages %s %s>' % (ns, resultArgs)
else:
logging.info('UNSUPPORTED INCLUDE "%s"' % args)
return '/* Unsupported Include: %s */' % args
def fullsearch(args):
#args=None >> {searchform ns=}
#args='' >> {{backlinks>.}}
#args!='' >> {{search><args>}}
#ignore special searches. see MoinMoin page "HilfeZumSuchen"
if args is None:
return '{searchform ns=}'
elif ':' in args or ' ' in args:
logging.info('UNSUPPORTED SEARCH %s(%s)' % (name, args))
return '/* Unsupported Search %s(%s). may be backlinks plugin will help */' % (name, args)
elif args=='':
return '{{backlinks>.}}'
elif name=='PageList':
return '{{backlinks>%s}}' % ":".join(args.split('/')).lower();
else:
logging.info('UNSUPPORTED SEARCH %s(%s)' % (name, args))
return '/* Unsupported Search %s(%s) */' % (name, args)
try:
lookup = {
'BR' : ' \\\\ ',
'br' : ' \\\\ ',
'MailTo' : email,
'GetText' : args,
'ShowSmileys' : inherit,
'ShowAttachedFiles' : showAttachedFiles,
'Include' : includeMacro,
#no real fulltext search!
'FullSearch' : fullsearch,
'FullSearchCached' : fullsearch,
'PageList' : fullsearch,
'MonthCalendar' : monthcal,
'Navigation' : navigation,
'TableOfContents' : '',
'RandomQuote': randomQuote,
'Anchor': inherit,
'Action': inherit,
'Icon': inherit,
'FootNote': footnote,
'Date': dateMacro,
'DateTime': dateTimeMacro
}[name]
except KeyError:
logging.info('UNDEFINED MACRO "%s"' % name)
lookup = '/* UndefinedMacro: %s(%s) */' % (name, args)
if type(lookup) == FunctionType:
text = lookup(args)
else:
text = lookup
return text
def smiley(self, text):
try:
# https://www.dokuwiki.org/devel:smileys.conf
return {
# note: reverse sorted so that longer smileys get matched first
'X-(' : ':-X',
'{X}' : ':!:',
'{*}' : '<ubu>',
'(./)' : u'',
':))' : ':-P',
':-))' : ':-P',
':-?' : ':-P',
':o' : ':-o',
'{OK}' : ':!:',
'{o}' : '<circ>',
'{i}' : ':!:',
':D' : ':-D',
'B)' : '8-)',
'B-)' : '8-)',
'{3}' : '<3>',
'{2}' : '<2>',
'{1}' : '<1>',
'(!)' : ':!:',
'/!\\' : ':!:',
':\\' : ':-\\',
':))' : ':-)',
':)' : ':-)',
':(' : ':-(',
':-))' : ':-)',
':-)' : ':-)',
':-(' : ':-(',
';)' : ';-)',
'|)' : ':-|',
'|-)' : ':-|',
'>:>' : '^_^',
'<!>' : ':!:',
'<:(' : ':-?',
}[text]
except KeyError:
return text