Not to rain on your parade, but I'm going to, anyway. Isn't the idea of science fiction to entertain? There has to be conflict, story arcs, have to be characters we care about. As long as we're imagining things, time travel is so much more amenable to an engaging work of fiction.
This idea would make a great SNL skit though...
"Behold! My new invention! The past-peeker!!!!!!"
"Ohhh we can travel back and forward in time!"
"Ummm no, you can't travel into the future....."
"But I can still walk the streets of London in 1670?"
"Well, no....but....you can LOOK AT THEM!!!!!"
"So, I can see into the future, then?"
"Look, you can see the past, OK?"
Crowd dissipates, except for pervert from back of crowd, who conspires with inventor in a whisper,
"I can watch anything, anywhere from the past, without being detected?"
(uneasily) "Ummm, yes...?"
Pervert looks exultant. Inventor defeated as he takes the $20 offered.