Not needing turn 1-2 hate cards is "very casual"? Low bar.
Much like sol ring, I think library is totally fine in a casual deck. It greases the wheels of your deck, but it doesn't win the game. If you're just using it to draw more 75% cards it's not going to be much of a problem. Although ofc, sol ring is a couple bucks so it goes everywhere. Library is expensive enough it'd likely show up primarily in high powered decks where it's more threatening.
I'd assume most casual groups would ignore it getting unbanned since nobody would have one. And if they did, and put it into a casual deck, I don't think it'd make it "un-casual" all of a sudden.