Learning To Interact With The 3D World