I know this isn't the answer you want, but consider switching away from compose entirely
A local kubernetes instance handles all the routing for me, and since i was using that anyway podman was legitimately a drop in replacement for docker.
Podman is just the tool that creates the container for me, running it gets handled by something else entirely.