Time is our most precious currency. It makes us choose, select, edit. What stays? What goes?
Who are we, in the end? The way we spend our most valuable currency--our time.
To put it bluntly, if you think you have to choose between the challenge of making English your own and your other passions--you'll have to pick one or the other.
The way I see it, you have to choose both.
If you love books, if you can't imagine life without reading, then pick up your book in English and you've just found a way to engage two passions at once--to beat time.
The same is true for your reader's journal. Incorporate more of what you love into what you're already doing.