SciTELuaExporters

Introduction

SciTELuaExporters is a set of Lua scripts for use with SciTE's Lua extension (present in SciTE 1.60 and later) that more or less duplicates the functionality of the standard set of SciTE file exporters. These scripts were adapted from C++ code in SciTE (in src/Exporters.cxx), originally in support of a project to develop a feature-ful PDF exporter for SciTE. The planned PDF exporter is in development (update: this is in hiatus, because the ODT exporter currently meets my requirements quite well,) but in the meantime, I ported over all the exporters (for HTML, PDF, RTF, TeX and XML) to give myself some experience of writing SciTE extensions in Lua. In addition, there is a greatly enhanced PDF exporter, and three new exporters: ABW (Abiword), SXW (OpenOffice.org), ODT (Open Document).

These scripts are not meant to replace the existing exporters, but to supplement them for advanced SciTE users. They are also a kind of proof-of-concept of writing sophisticated scripts using the Lua extension. You might find these scripts useful if:

• You want to customize or extend a particular exporter without recompiling SciTE or maintaining your own customized version.
• You want to quickly develop or prototype a new file exporter.
• You want to be able to export selections of text or code, or do things in Lua without changing the standard SciTE distribution.

SciTELuaExporters uses the same MIT-style license as SciTE, and is being distributed in the spirit of sharing. Should you find these scripts useful, consider sharing your code as well under the terms of the MIT license, or linking back to this package, or sharing your experience. New exporters are welcome; there may be practical limits to how many file exporters one can put in SciTE, but there are no such limits to user-selectable and user-customizable Lua scripts. The basic set of exporters will be kept similar to their compiled counterparts, and modified versions will be maintained as separate files.

Download

Main links: Project Home Page on LuaForge | Project Summary Page on LuaForge | LuaForge

File Releases: download SciTELuaExporters-0.9.11.zip (98KB).

This release should be considered to be of beta quality. They have been tested with several lexers, and are quite robust. A 1.0 release will be made later, when the exporters have got a bit more of a workout. Note that the current exporters are a little behind the SciTE exporters, that is, they are still based on the Exporters.cxx file from around 2006. This will be rectified later.

What's new?

• [0.9.11] Everything updated to Lua 5.1, targeting SciTE 1.74 or later.
• [0.9.10] Added windows-1252 to UTF-8 character set translation for ABW, ODT and SXW.
  
• [0.9.10] Updated to match SciTE 1.68. Fixed whitespace glitch for ODT and SXW.
  
• [0.9.9] New ODT (Open Document) exporter, output tested on OOo 2beta
• [0.9.9] SXW exporter fixed to enable output files to be loaded by OOo 2beta
  
• [0.9.9] Updated files to SciTE 1.63. Fixed line numbering handling for PDF+ exporter
  
• [0.9.8] New ABW (AbiWord) exporter for AbiWord 2.2, same features as SXW exporter
• [0.9.8] Added page types to SXW exporter
  

User's Guide

Note: Please see the file SciTELuaExporters.lua if you want something that can run immediately.

You need to be running a recent version of SciTE. The scripts were developed to run on a self-compiled vanilla Win32 SciTE that roughly tracks the official releases. I don't use Linux that much, so I haven't had much chance to test the scripts extensively on the Linux version of SciTE. If you are a serious user of SciTE Lua scripts, I recommend placing all Lua scripts in a separate subdirectory in your Win32 SciTE installation. Then, put the following line (or something like it) in your user properties file:

  ext.lua.startup.script=$(SciteDefaultHome)/script/SciTEStartup.lua

Keeping all Lua scripts in a /script directory will make maintenance easier. Please read the SciTE documentation (SciTEDoc.html) for the details on configuration. Upgrading simply means copying the /script directory and your user properties file over, it's that simple. There are 8 separate exporters, for ABW, HTML, ODT, PDF, RTF, SXW, TeX and XML. Some exporters have an extended version with additional features,

Step 1 is to load the scripts. You can place the following in your SciTEStartup.lua file to make loading of scripts easier:

The function will load a given script filename from the /script directory. If all your SciTE Lua scripts are in the same directory, then the usual Ctrl+Shift+O will very quickly open the script file under the caret.

The two lines dealing with "Charset" is for ABW, ODT and SXW exporters only. If you are not using one of these exporters, you can safely delete those lines. The first line initializes the module for translating 8-bit character sets into UTF-8 for the three exporters mentioned. The second line sets the source character set. By default, the translation is for "iso-8859-1". For Windows users, "windows-1252" is available.

Step 2 is to install an exporter function as a shortcut. When developing Lua-scripted SciTE functions, I add the following in SciTEUser.properties:

  command.name.1.*=Experimental Script
  command.subsystem.1.*=3
  command.1.*=experimental
  command.save.before.1.*=2

This will bind the Lua function experimental() to Ctrl-1. Then you can call the exporter function you want by writing a function like this in your SciTEStartup.lua file:

  function experimental()
    exporters:SaveToHTML()
  end

By default, an exporter will open the output window and print a helpful message, including the full path of the file it is writing. Exporters will never overwrite any existing file, but will append a number to the exported filename to create a new, unique filename. Be warned, though, that exporting very large files takes several seconds, longer still on slow machines. You can force exporters to be silent using:

  ext.lua.exporters.verbose=0

The exporters should be quite robust. If they crash, please let me know.

Files in the Distribution

The following are non-script files:

• SciTELuaExporters.html: This document.
  
• Changelog_SciTELuaExporters: What has been changed.
• Manifest_SciTELuaExporters: List of files in this distribution.
• pdf_linenumbers.diff: Example of a feature. Line numbering for the PDF exporter.

The following are the base script files:

• SciTE_ExportBase.lua: Utility framework for exporting. Useful functions to simplify the coding of exporters. Must be loaded before any of the following exporters.
  
• SciTE_ExporterTemplate.lua: Basic public domain template for developers.
  
• SciTE_ExporterHTML.lua: Export to HTML.
• SciTE_ExporterPDF.lua: Export to PDF.
• SciTE_ExporterRTF.lua: Export to RTF.
• SciTE_ExporterTeX.lua: Export to TeX.
• SciTE_ExporterXML.lua: Export to XML.

The following are extended script files:

• SciTE_ExporterHTMLPlus.lua: HTML exporter with the following extended features:
• Disable background highlights.
• Auto-enables UTF-8 encoding in HTML if document is encoded as such.
  
• SciTE_ExporterPDFPlus.lua: PDF exporter with the following extended features:
• Line numbering.
  
• Line spacing.
• Line wrapping for Helvetica and Times fonts in addition to Courier wrapping.
• Pairwise kerning for Helvetica and Times, but without ligatures.
• Warning! You might find the script slow for large files.
• SciTE_ExporterSXW.lua: SXW exporter (OpenOffice.org) with the following features:
• Output tested on OpenOffice.org 1.1.2 and OpenOffice.org 2beta on Win32.
  
• Export UTF-8 documents to be read by OpenOffice.org. 8-bit encodings may have to be recoded. No support for UCS-2 yet.
  
• Export named styles to the OpenOffice.org Stylist for easy formatting changes.
• Exact font specifications and background colours (but no eolfill.)
• Set tab stops, paper orientation, paper size by name, and page margins.
• Lots more customization can be done through a variety of properties.
• Mono font override (for monospace too) and font substitution.
• Warning! The script is quite slow if unaccelerated (see next item), about 5-10K/sec source processing rate on an Athlon 2500+.
• Can transparently detect and use Reuben Thomas' bitwise function library to speed up CRC calculation by 2X to 3X, giving source processing rates of 15-25K/sec. Currently, the library can be found in Bruce Dodson's version of SciTE.
• Optional automatic translation of 8-bit encodings (e.g. windows-1252) to UTF-8.
  
• SciTE_ExporterABW.lua: ABW exporter (AbiWord):
  
• Output tested on AbiWord 2.2.2 for Win32.
• Roughly the same feature set as the SXW exporter.
• UTF-8 ready. 8-bit encodings may have to be recoded. No support for UCS-2 yet.
• Faster than SXW exporter because there is no need to wrap data into a Zip archive.
• Optional automatic translation of 8-bit encodings (e.g. windows-1252) to UTF-8.
• SciTE_ExporterODT.lua: ODT exporter (Open Document):
• Output tested on OpenOffice.org 2beta for Win32.
• Roughly the same feature set as the SXW exporter.
• UTF-8 ready. 8-bit encodings may have to be recoded. No support for UCS-2 yet.
• Warning! The script is quite slow due to CRC calculation (see the SXW exporter notes.)
• Optional automatic translation of 8-bit encodings (e.g. windows-1252) to UTF-8.
• SciTE_ExportCharset.lua: Implements 8-bit character set translation to UTF-8. This is not an exporter but a utility module. It is used by the ABW, ODT and SXW exporters.
  

Developer's Guide

SciTE_ExportBase.lua provides a number of functions, variables and constants to perform some common tasks required for exporting data. All functions, variables and constants are defined in two tables, exportutil, containing export utility functions, and stylemgr, containing style management functions. The exporters themselves are located in separate self-contained files, and each exporter creates a function in the exporters table. If these three tables collides with the Lua namespace in your SciTE setup, you will have to change the names of the tables.

SciTE_ExportTemplate.lua is a sample exporter that is public domain code, and is heavily commented for developers of Lua-based SciTE exporters.

Here is a summary of the functions found in SciTE_ExportBase.lua. Currently, the style manager can handle a single style set only. To see usage examples, look at the working exporters.

• exportutil.exportfile(filedir, basename, newext)
  Constructs a suitable filename from parts of a complete path. A potential filename is tested to see if it exists. If it exists, a number in parentheses is added and the new filename is tested. In order to avoid an endless loop, the numbering goes up to 999. The filename that is returned is ready to be opened and written to by the exporter.
  
• exportutil.progress(status)
  Replaces the current line in the output window. This can be used to indicate processing progress. The implemented exporters do not use this function at present because Lua is fast enough for most files. Writing several hundred KBs of output takes only a second or two on a fast machine. Doesn't work in practice because SciTE currently updates the output pane only after the script function is complete.
  
• exportutil.CharAt(pos)
  Wraps editor.CharAt[pos], because a signed 8-bit value is returned, with a range of -128 to +127. The function returns a single-character string instead of a numeric value, in the interest of code readability.
  
• exportutil.StyleAt(pos)
  Wraps editor.StyleAt[pos], stripping out higher-order bits that are not style bits. While this behaviour isn't common to all of the original C++ exporters, the Lua adaptations gives the same output as the original C++ exporters.
  
• exportutil.propbool(property, default)
  Helper function for property lookup. Looks up a property value used as a flag that is supposed to be set to a 0 or 1, and returns the result as a Lua boolean value. The argument default needs a numeric 0 or 1, not a boolean value.
  
• exportutil.GetSelPos()
  Helper function for text selection. Returns the beginning and end position selected text positions, in proper order. If no text is selected, then the range for the entire document is returned. The exporter is supposed to handle or trap the case of a zero-length document.
  
• stylemgr:setlexer(lexer)
  Sets the lexer by number for which the style information is to be parsed and processed. This number can be retrieved using editor.Lexer.
  
• stylemgr:getlexer()
  Gets the lexer by number for which the style information is to be parsed and processed.
• stylemgr:setlexerlanguage(lexerlanguage)
  Similar to stylemgr:setlexer, but requires a string identifying the lexer instead of a number. Currently, this function looks up the lexer number in a fixed table. If a lexer is added to Scintilla, the table will need to be updated.
  
• stylemgr:getlexerlanguage()
  Similar to stylemgr:getlexer, but returns a string identifying the lexer instead of a number. Uses the same fixed table. This function is used by getpropname to build the correct style property strings to look up once a lexer has been selected.
• stylemgr:getpropname(n)
  Makes a property name based on the style number given and the currently set lexer style. This is used to retrieve the style settings.
  
• stylemgr:initlexer(lexer)
  An internal function that initializes the default styles (32 to 37). Use stylemgr:setlexer or stylemgr:setlexerlanguage to initialize the style manager instead.
  
• stylemgr:locate(n)
  Returns a table containing the processed style information for the given style number based on the currently selected lexer style. If the style has already been located and processed, and is present in the list of styles, then that table is returned.
  
• stylemgr:retrieve(property, base)
  An internal function used to parse and process style information, and to set the appropriate fields in a style table for the exporter to use. Also handles inheritance of styles given a base style in the argument list. Use directly if you want to write code that can override the processing done by stylemgr:locate, for example, style overrides for a specific exporter. You can also use this function to implement non-standard inheritance behaviour.
  
• stylemgr:hexstr(colour)
  Convenience function to return a colour specification in #RRGGBB format, given a colour component table used in a style table.
  
• stylemgr:showstyle(style)
  Used for debugging. Given a style table, returns a formatted string of style fields. You can write the information in the editor window using editor:AddText.
  
• stylemgr:showlexer(lexer)
  Used for debugging. Given a lexer number or name, returns a formatted string of styles.
• stylemgr:showeverything()
  Used for debugging. Returns a formatted string of most styles of all lexers.

The exporters that are currently implemented in this distribution are:

• exporters:SaveToABW()
  
• exporters:SaveToHTML()
• exporters:SaveToODT()
• exporters:SaveToPDF()
• exporters:SaveToRTF()
• exporters:SaveToSXW()
• exporters:SaveToTeX()
• exporters:SaveToXML()
  

Step 1 is to initialize the style manager using the current lexer number or name:

Step 2 is to grab the styles you want to access using the stylemgr:locate function, then either assigning the return style table to your own variable, or access the list of styles after calling the function. The former is done like this:

The style table can be accessed in a variety of methods. Use whatever method you like, keeping in mind readability is important when writing a script you expect others to use and extend. For example:

The global default styles are created when the style manager is initialized, as the list stylemgr.basestyles, as opposed to the style list for the current lexer, which is processed on-demand, and is named stylemgr.styles. The default style for each list is always the usual style 32. The other predefined styles that have specific usage are numbered 33 to 37. I don't think they appear when processing a normal document, but you can access and use them if you want to, using the usual stylemgr:locate function.

Style information is returned in a table with all field elements always filled in. If a property does not explicitly specify a field, then that field is inherited from the base style (style 32). Fields that are explicitly specified have their field names added to a string, the specified field. This is probably not a very high-performance method but it works well and is readable.

The table fields are as follows:

• font: a string, e.g. "Courier New"
• size: a number, e.g. 12
  
• case: a string, either "m", "u" or "l"
  
• fore: a table with fields "r", "g" and "b" which are numbers in the range 0-255
  
• back: a table with fields "r", "g" and "b" which are numbers in the range 0-255
• italics: a boolean
  
• bold: a boolean
• eolfilled: a boolean
• underlined: a boolean
  
• visible: a boolean
• changeable: a boolean
• specified: a string, e.g. if only the size and font is specified in the property string, then specified is defined as "font size "
  

An exporter should accept an editor start position and an end position. By using editor positions, an exporter can easily export the whole file or a selected part of it. exportutil.GetSelPos() helps with getting the position information during execution. If you need to process the document line-by-line, you can retrieve the line number using the appropriate SciTE API calls, like editor:GetLine. For more on writing exporters, look at the completed ones found in this package.

pdf_linenumbers.diff is an example of a feature. The diff adds optional line numbering to the PDF exporter, all in less than 20 lines of Lua code and less than half an hour's work. The upshot of it is that you do not have to customize and recompile the SciTE binaries; you just need to move a portable custom Lua script when upgrading.

Acknowledgements

SciTE is principally developed by Neil Hodgson, with the help of many contributors, and can be found at: http://scintilla.sourceforge.net/SciTE.html. Bruce Dodson did much of the work to make the Lua extension usable. Thanks to the LuaForge team for hosting this material. This page was written on Mozilla. SciTELuaExporters was developed on SciTE, which I compiled on MinGW, thanks to GNU software.

I consider the basic set of Lua exporters a straightforward conversion from C/C++ code, so I decline to claim them as my own code. Neil Hodgson's Scintilla/SciTE license applies to these scripts. The code for the Lua exporter support routines (SciTE_ExportBase.lua) and other exporters are either copyrighted by me or their respective authors, and the same MIT-style license applies. You can basically do almost anything you like to the scripts and this documentation except claim it as your own. The license text can be found at the beginning of each script.