I spent a couple of years living in East London, for my sins (as close in as Bethnal Green and as far out as Walthamstow) and it IS a strange place, although the twin factors of urban decay and hipsterness rub uncomfortably against each other a bit too much and take the edge off any real psychogeographical air, IMO. Shame, really. Anyway, one of the more interesting things to do is walk the length of the road that goes from Aldgate East to, well, Romford or something - not the whole way, obviously. You can stroll down from Hackney to Whitechapel (maybe taking in Spitalfields, although the air has been sucked out of that place), turn left up the road towards Mile End - and then walk to Stratford. This takes in all sorts of really interesting sights - you go past some interesting and historical pubs (The Blind Beggar, The Grave Maurice), some interesting squares and gardens (the area around Mile End and Queen Mary's, with the landscaped nature garden and the lovely housing squares), some interesting developments, (that arts trust up in Bow, near the church) and end up at the Olypmic developments. Doing this you get a real sense for the way that one road can connect a lot of parts of the city: the area by Bow Church in particular makes you feel as if you're stuck in between two very distinct zones, the really noisy and busy parts of Whitechapel, Mile End, Stepney Green and the developing sections of Stratford and Newham. It's also really interesting to see how they're adapting the waterways (the Bow canals, Fish Island and so on) for the Games.
Little pockets of Hackney are good to explore - like the village near Victoria Park, which is now quite swanky, and the park itself - with the pavilion and lurking pubs. There's also a completely barmy pub in a park next to Grove Road (just about where it joins with the Roman Road) which isn't weird in itself, it just sticks out of the landscape like a rotten tooth. It's always full of interesting characters with a couple of stories to tell too, and it's called the Palm Tree. It's fitting that pubs shape a lot of the landscape of this part of London, I reckon, given that they play such a huge part in the community and have done for centuries.
The liberal scattering of tabernacles, Orthodox churches, candle factories, playhouses, steam baths, music halls, crime scenes, listed public toilets and mosques also gives the area a very unusual and fascinating character.
There's a pub on Bow Road called the Bow Bells (I think) which has a sign above the gents reading "Beware of the ghost" but I never found out what that was about.
And on yet another pub theme, try The Ten Bells down near Brick Lane, if you haven't already: a couple of Jack The Ripper's victims used to drink in there. In fact, if you feel like exploring Whitechapel and Aldgate a bit more, have a read of Donald Rumbelow's Jack The Ripper book, and Slow Chocolate Autopsy by Ian Sinclair. Hawksmoor by Peter Ackroyd also fictionalises the supposedly Satanic influence on a lot of the area's churches.