I would mark this as a bug. But my issue template was not accepted. Couldn't figure out why, didn't think it matters much either.
This is a problem in the latest version, possibly all versions with --header.

Current Behaviour

lsd -l --header doesn't filter ANSI sequences when not on a tty/pty
lsd -l --header | less for example

Expected Behaviour:

It should filter ANSI sequences.