What you are saying is correct, and why I am looking for a better way.
It was just simpler for me to set up a complete image that can be used on a VM.

I have seen other similar projects where the setup is a hassle, which is why I am looking into it.

(changing requirements, custom scripts, extension)

I did it to save on space, LXDE could also be an option but with a bit more space.

For people that don’t get the joke: this messes up your system