Electrical tape sometimes leaves adhesive behind when you peel it off, so you might not be able to avoid using a chemical during the reversal process.

Yeah, but electrical tape looks like garbage. Trade off is aesthetics vs ability to remove easily.

There are various sizes/shapes of stickers intended to either dim or block led lights if aesthetics are important.