You can only get AVCameraCalibrationData from depth data output or image output.
But if all you need is FOV, you only need some of the information that this class offers – the camera's unique matrix – and you can get it by itself from AVCaptureVideoDataOutput.
Set cameraIntrinsicMatrixDeliveryEnabled on AVCaptureConnection connects the camera to the recording session. (Note that you should check cameraIntrinsicMatrixDeliverySupported first, not all recording formats support intrinsic.)
When the video output turns test buffers, check each sample buffer attachment for the kCMSampleBufferAttachmentKey_CameraIntrinsicMatrix key. As mentioned in CMSampleBuffer.h (someone is archiving a radar to get this information in the online documentation), the value of that attachment is a CFData encoding for a matrix_float3x3, and the elements (0,0) and (1