My concerns have been perhaps:
Unable to share a book with other people.
Can they be viewed on multiple sources by the same owner? Like Kindle, tablet, computer?
They may be ok for novels but for guidebooks or technical books, even if I electronically scribble on them, the may not look as good as with a pen or fold the pages and get to the page(s) quickly.
And; the unsureness of keeping that e-book for many years ie - if the e-reader breaks or is discontinued/unsupported, as well as will the old e-books still be readable in future readers and can one backup e-books and reinstall them?