There definitely should be a good open source e-reader, but for what it’s worth I use a Kobo Clara 2e (newer models are available in both black and white and colour eink) and it works fine for me.
I download books from various resources; like Project Gutenberg and use Calibre for managing them. Works pretty seamlessly, especially with the Calibre Kobo plugin for automatic conversion to the kepub format too. However this obviously requires the use of a computer, which may be a dealbreaker. Also Kobo works well with Overdrive for borrowing library ebooks, which is neat.