You have answered the question yourself, you must have had the rear camera pointing at the covered part of the FPV goggles, try rotating the phone 180deg in the case, the goggles have a cutout on the right hand side that allows the camera to see.
I just got the Bebop 2 FPV pack and I've been having a blast with it. I was reading the manual and found that the Right Trigger does a switch between Direct and Immersive view. When I press the button while in headset view, the picture goes completely black (it's got the UI lines but no picture).
I was wondering what the deal with that is, or if I'm missing anything here.