Structured 3D Representations For Inferring Geometry And Interaction