So lesson learned. After spending many hours sanding and polishing the chassi plate to a near mirror finish I came to the conclusion that it had to be painted instead. The aluminium is soft, very soft, and therefore very very sensitive to scratches. It was almost impossible to handle it without leaving small marks on it. Just placing the volume knob would dent it...
So doing the assembly and soldering without leaving scuff marks and scratches felt like a "mission impossible". This was a hard call, I really really liked the mirror finish and worked pretty hard to get there, but in the end it had to work in everyday life.
I gave the plate a coat of matte black acrylic and then a few thin layers of car paint. Real car paint (bought at a paint shop) is great since it comes without any clear coat and creates a finish similar to what you get when anodizing metal. Of course, when using it on cars you need a clear coat, but in this case I wanted that anodized look.
On my next build I will get a stainless chassi plate cut in a water jet instead.