Pandoc Lua filter microbenchmarks

Here, we’ll use the largest synthetic file from the previous note, with 4096 paragraphs.

Running a no-op filter repeatedly

Consider this filter:

local n

function Meta(meta)
    n = tonumber(pandoc.utils.stringify(meta.n_times)) or 1
end

function Pandoc(doc)
    for i = 1, n do
        doc.blocks = doc.blocks:walk({})
    end
    return doc
end

We run it like so:

import utils
for n in [10, 100, 1000]:
    print("%s repetitions: %.3fs s" % (n, utils.time_call("quarto pandoc ../_supporting_docs/synth-benchmark-1/inputs/_size-12.qmd -L ../_supporting_docs/filters/nopwalker.lua -M n_times=%s -f markdown -t json -o /dev/null" % n)))

10 repetitions: 1.327s s
100 repetitions: 1.433s s
1000 repetitions: 2.461s s

Clearly, each repetition takes negligible time.

Running a read-only filter repeatedly

On the other hand, consider this:

local n

function Meta(meta)
    n = tonumber(pandoc.utils.stringify(meta.n_times or "1")) or 1
end

function Pandoc(doc)
    local c = 0
    for i = 1, n do
        doc.blocks = doc.blocks:walk({
            Str = function(el)
                c = c + 1
            end
        })
    end
    -- print(c)
    doc.blocks = pandoc.Blocks({})
    return doc
end

This takes significantly longer:

for n in [1, 10]:
    print("%s repetitions: %.3fs s" % (n, utils.time_call("quarto pandoc ../_supporting_docs/synth-benchmark-1/inputs/_size-12.qmd -L ../_supporting_docs/filters/readonlywalker.lua -M n_times=%s -f markdown -t json -o /dev/null" % n)))

1 repetitions: 1.832s s
10 repetitions: 7.707s s

If we implement the filter entirely in Lua, this gets significantly faster (!):

local n

function Meta(meta)
    n = tonumber(pandoc.utils.stringify(meta.n_times or "1")) or 1
end

function pandoc_type(el)
    local result = el.tag
    if result == "table" or result == nil then
        return pandoc.utils.type(el)
    end
    return result
end

function lua_readonly_walk(el, filter)
    local t = pandoc_type(el)
    if filter[t] then
        filter[t](el)
    end
    local n = 0
    local c = nil
    if t == "Blocks" or t == "Inlines" then
        c = el
        n = #el
    elseif el.content then
        c = el.content
        n = #el.content
    end
    for i = 1, n do
        lua_readonly_walk(c[i], filter)
    end
end

local profiling = false
local profiler = dofile('profiler.lua')
function Pandoc(doc)
    if profiling then
        os.execute('rm -f output.prof')
        profiler.start('output.prof', 1)
    end
    local c = 0
    for i = 1, n do
        if profiling then
            profiler.setcategory('pass-' .. i)
        end
        lua_readonly_walk(doc.blocks, {
            Str = function(el)
                c = c + 1
            end
        })
    end
    print(c)
    doc.blocks = pandoc.Blocks({})
    if profiling then
        profiler.stop()
    end
    return doc
end

for n in [1, 10]:
    print("%s repetitions: %.3fs s" % (n, utils.time_call("quarto pandoc ../_supporting_docs/synth-benchmark-1/inputs/_size-12.qmd -L ../_supporting_docs/filters/readonlypureluawalker.lua -M n_times=%s -f markdown -t json -o /dev/null" % n)))

Error running filter ../_supporting_docs/filters/readonlypureluawalker.lua:
cannot open profiler.lua: No such file or directory
stack traceback:
    ../_supporting_docs/filters/readonlypureluawalker.lua:35: in main chunk
Error running filter ../_supporting_docs/filters/readonlypureluawalker.lua:
cannot open profiler.lua: No such file or directory
stack traceback:
    ../_supporting_docs/filters/readonlypureluawalker.lua:35: in main chunk
Error running filter ../_supporting_docs/filters/readonlypureluawalker.lua:
cannot open profiler.lua: No such file or directory
stack traceback:
    ../_supporting_docs/filters/readonlypureluawalker.lua:35: in main chunk
Error running filter ../_supporting_docs/filters/readonlypureluawalker.lua:
cannot open profiler.lua: No such file or directory
stack traceback:
    ../_supporting_docs/filters/readonlypureluawalker.lua:35: in main chunk
Error running filter ../_supporting_docs/filters/readonlypureluawalker.lua:
cannot open profiler.lua: No such file or directory
stack traceback:
    ../_supporting_docs/filters/readonlypureluawalker.lua:35: in main chunk

1 repetitions: 1.171s s

Error running filter ../_supporting_docs/filters/readonlypureluawalker.lua:
cannot open profiler.lua: No such file or directory
stack traceback:
    ../_supporting_docs/filters/readonlypureluawalker.lua:35: in main chunk
Error running filter ../_supporting_docs/filters/readonlypureluawalker.lua:
cannot open profiler.lua: No such file or directory
stack traceback:
    ../_supporting_docs/filters/readonlypureluawalker.lua:35: in main chunk
Error running filter ../_supporting_docs/filters/readonlypureluawalker.lua:
cannot open profiler.lua: No such file or directory
stack traceback:
    ../_supporting_docs/filters/readonlypureluawalker.lua:35: in main chunk
Error running filter ../_supporting_docs/filters/readonlypureluawalker.lua:
cannot open profiler.lua: No such file or directory
stack traceback:
    ../_supporting_docs/filters/readonlypureluawalker.lua:35: in main chunk

10 repetitions: 1.189s s

Error running filter ../_supporting_docs/filters/readonlypureluawalker.lua:
cannot open profiler.lua: No such file or directory
stack traceback:
    ../_supporting_docs/filters/readonlypureluawalker.lua:35: in main chunk