如何访问 Hololens 前置摄像头

Question

我正在使用全息镜头，我正在尝试获取前置摄像头的图像。我唯一需要的是拍摄该相机的每一帧图像并将其转换为字节数组。

Answer 1

1. 遵循文档中的 Unity 示例。它有一个写得很好的例子： https://docs.unity3d.com/Manual/windowsholographic-photocapture.html

从上面链接的统一文档中复制：

using UnityEngine;
using System.Collections;
using System.Linq;
using UnityEngine.XR.WSA.WebCam;

public class PhotoCaptureExample : MonoBehaviour {
PhotoCapture photoCaptureObject = null;
Texture2D targetTexture = null;

// Use this for initialization
void Start() {
    Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
    targetTexture = new Texture2D(cameraResolution.width, cameraResolution.height);

    // Create a PhotoCapture object
    PhotoCapture.CreateAsync(false, delegate (PhotoCapture captureObject) {
        photoCaptureObject = captureObject;
        CameraParameters cameraParameters = new CameraParameters();
        cameraParameters.hologramOpacity = 0.0f;
        cameraParameters.cameraResolutionWidth = cameraResolution.width;
        cameraParameters.cameraResolutionHeight = cameraResolution.height;
        cameraParameters.pixelFormat = CapturePixelFormat.BGRA32;

        // Activate the camera
        photoCaptureObject.StartPhotoModeAsync(cameraParameters, delegate (PhotoCapture.PhotoCaptureResult result) {
            // Take a picture
            photoCaptureObject.TakePhotoAsync(OnCapturedPhotoToMemory);
        });
    });
}

void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame) {
    // Copy the raw image data into the target texture
    photoCaptureFrame.UploadImageDataToTexture(targetTexture);

    // Create a GameObject to which the texture can be applied
    GameObject quad = GameObject.CreatePrimitive(PrimitiveType.Quad);
    Renderer quadRenderer = quad.GetComponent<Renderer>() as Renderer;
    quadRenderer.material = new Material(Shader.Find("Custom/Unlit/UnlitTexture"));

    quad.transform.parent = this.transform;
    quad.transform.localPosition = new Vector3(0.0f, 0.0f, 3.0f);

    quadRenderer.material.SetTexture("_MainTex", targetTexture);

    // Deactivate the camera
    photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}

void OnStoppedPhotoMode(PhotoCapture.PhotoCaptureResult result) {
    // Shutdown the photo capture resource
    photoCaptureObject.Dispose();
    photoCaptureObject = null;
}
}

获取字节：替换以下方法：

    void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame) {
    // Copy the raw image data into the target texture
    photoCaptureFrame.UploadImageDataToTexture(targetTexture);

    // Create a GameObject to which the texture can be applied
    GameObject quad = GameObject.CreatePrimitive(PrimitiveType.Quad);
    Renderer quadRenderer = quad.GetComponent<Renderer>() as Renderer;
    quadRenderer.material = new Material(Shader.Find("Custom/Unlit/UnlitTexture"));

    quad.transform.parent = this.transform;
    quad.transform.localPosition = new Vector3(0.0f, 0.0f, 3.0f);

    quadRenderer.material.SetTexture("_MainTex", targetTexture);

    // Deactivate the camera
    photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}

用下面的方法：

void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
    List<byte> imageBufferList = new List<byte>();
    imageBufferList.Clear();
    photoCaptureFrame.CopyRawImageDataIntoBuffer(imageBufferList);
    var bytesArray = imageBufferList.ToArray();
}

1. 如果您现在使用 Unity 2018.1 或 2018.2（我认为也是 2017.4），那么它将无法正常工作。 Unity 有公共跟踪器来解决它： https://issuetracker.unity3d.com/issues/windowsmr-failure-to-take-photo-capture-in-hololens
2. 我创建了一个解决方法，直到 Unity 修复了该错误： https://github.com/MSAlshair/HoloLensMediaCapture

不使用来自 unity 的 PhotoCapture 作为解决方法的基本示例：以上链接中的更多详细信息

您必须添加 #if WINDOWS_UWP 才能使用 MediaCapture： 理想情况下，您希望从 unity 中使用 PhotoCapture 来避免这种情况，但在 Unity 解决问题之前，您可以使用类似的东西。

#if WINDOWS_UWP
        public async System.Threading.Tasks.Task<byte[]> GetPhotoAsync()
        {
            //Get available devices info
            var devices = await Windows.Devices.Enumeration.DeviceInformation.FindAllAsync(
                Windows.Devices.Enumeration.DeviceClass.VideoCapture);
            var numberOfDevices = devices.Count;

            byte[] photoBytes = null;

            //Check if the device has camera
            if (devices.Count > 0)
            {
                Windows.Media.Capture.MediaCapture mediaCapture = new Windows.Media.Capture.MediaCapture();
                await mediaCapture.InitializeAsync();

                //Get Highest available resolution
                var highestResolution = mediaCapture.VideoDeviceController.GetAvailableMediaStreamProperties(
                    Windows.Media.Capture.MediaStreamType.Photo).
                    Select(item => item as Windows.Media.MediaProperties.ImageEncodingProperties).
                    Where(item => item != null).
                    OrderByDescending(Resolution => Resolution.Height * Resolution.Width).
                    ToList().First();

                using (var photoRandomAccessStream = new Windows.Storage.Streams.InMemoryRandomAccessStream())
                {
                    await mediaCapture.CapturePhotoToStreamAsync(highestResolution, photoRandomAccessStream);

                    //Covnert stream to byte array
                    photoBytes = await ConvertFromInMemoryRandomAccessStreamToByteArrayAsync(photoRandomAccessStream);
                }
            }
            else
            {
                System.Diagnostics.Debug.WriteLine("No camera device detected!");
            }

            return photoBytes;
        }

        public static async System.Threading.Tasks.Task<byte[]> ConvertFromInMemoryRandomAccessStreamToByteArrayAsync(
            Windows.Storage.Streams.InMemoryRandomAccessStream inMemoryRandomAccessStream)
        {
            using (var dataReader = new Windows.Storage.Streams.DataReader(inMemoryRandomAccessStream.GetInputStreamAt(0)))
            {
                var bytes = new byte[inMemoryRandomAccessStream.Size];
                await dataReader.LoadAsync((uint)inMemoryRandomAccessStream.Size);
                dataReader.ReadBytes(bytes);

                return bytes;
            }
        }
#endif

不要忘记向清单添加功能：

1. 网络摄像头
2. 麦克风：我不确定是否需要，因为我们只是拍照，但我还是添加了它。
3. 如果你想保存到图片库

Answer 2

在播放器设置中，启用对相机的访问，然后重试。