Why does a filter that is only supposed to change the alpha channel affect the RGB channels as well?

According to the docs, SVG filters are applied on pre-multiplied RGBA values. This means that the color components have been multiplied by the alpha component. When feComponentTransfer is applied, the color channels are un-pre-multiplied in order to apply the filter.

Pre-multiplying the RGBA values creates rounding errors:

#00171F 0.01 = #000000 (black)

#00171F 0.02 = #000001 (dark blue)

#00171F * 0.03 = #000101 (dark greenish blue)

So after pre-multiplication, the colors are quite different, but they don't contribute a lot to the final image, so it doesn't matter.

feComponentTransfer, however, will undo the pre-multiplication before it's applied, and then redo it when it's done. Because you are inverting the alpha channel, you end up greatly amplifying the color differences:

#000001 / .02 = #000032

#000101 / .03 = #002121

When the pre-multiplying is undone, the resulting differences are very significant.