still no word on how to convert/train other finetuned models into their format :(

APU board? They are going EOL soon, but these devices are built like a tank. Full Linux x86_64 support, coreboot bios. https://www.pcengines.ch/apu.htm A few sellers in the EU still have them.

Do not expect you can offer this service for a competive price against cloud prices. Caring for a company IT system is a big challenge and requires more work the more users there are.

For a company this size: make a clear contract. Consider how much time you need for setup/installation, monthly hours for maintenance, monitoring and at least daily(!) backups. Let them choose if they want it with a failover and charge for the required hours and material. Also put in the contract when they can expect support from you, including a clause for a holiday substitute admin (if needed). Then put a pricetag on support hours for holding people’s hands when they “can’t find that file they uploaded a week ago and it is surely a server issue” and put a pricetag on engineering hours for any modifications they might want, like installing any plugins they deem useful for themselves. Hardware prices, traffic, rack space and power should be included as well. Have a good plan for updates, choose your distro wisely, do not rely on autoupdates.

Play all this through in your head, add up the hours, choose a fair rate and then you have your pricetag.

Cloud will always be cheaper, because they have their infrastructure already deployed. Building from the ground up is more expensive, but I think it is worth it. Will they?