Yes you are able to walk through it, its like a glass/one-sided mirror. They look like prison bars on one side(you cant see through that side) and a rainbow shimmer from the other side, from which you can see through.
They are scattered on several locations through the map and works as assets to diversify gameplay and make it more engaging, adding an interesting mechanic.