StartCoroutine() 修复 targetTexture.ReadPixels 错误

Question

正如标题所暗示的那样，我在行发生错误时遇到了问题

targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);

错误：

ReadPixels 被调用以从系统帧缓冲区中读取像素，而 不在绘图框内。 UnityEngine.Texture2D:ReadPixels(矩形, Int32, Int32)

正如我从其他帖子中了解到的那样，解决此问题的一种方法是创建一个 Ienumerator 方法，该方法产生 return new WaitForSeconds 或其他内容，并将其称为：StartCoroutine(methodname) 以便及时加载帧，以便有将是像素来读取。

我不明白的是在下面的代码中，这种方法最有意义的地方。哪个部分没有及时加载？

    PhotoCapture photoCaptureObject = null;
    Texture2D targetTexture = null;
    public string path = "";
    CameraParameters cameraParameters = new CameraParameters();

private void Awake()
{

    var cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
    targetTexture = new Texture2D(cameraResolution.width, cameraResolution.height);

    // Create a PhotoCapture object
    PhotoCapture.CreateAsync(false, captureObject =>
    {
        photoCaptureObject = captureObject;
        cameraParameters.hologramOpacity = 0.0f;
        cameraParameters.cameraResolutionWidth = cameraResolution.width;
        cameraParameters.cameraResolutionHeight = cameraResolution.height;
        cameraParameters.pixelFormat = CapturePixelFormat.BGRA32;
    });
}

private void Update()
{
    // if not initialized yet don't take input
    if (photoCaptureObject == null) return;

    if (Input.GetKey("k") || Input.GetKey("k"))
    {
        Debug.Log("k was pressed");

        VuforiaBehaviour.Instance.gameObject.SetActive(false);

        // Activate the camera
        photoCaptureObject.StartPhotoModeAsync(cameraParameters, result =>
        {
            if (result.success)
            {
                // Take a picture
                photoCaptureObject.TakePhotoAsync(OnCapturedPhotoToMemory);
            }
            else
            {
                Debug.LogError("Couldn't start photo mode!", this);
            }
        });
    }
}

private static string FileName(int width, int height)
{
    return $"screen_{width}x{height}_{DateTime.Now:yyyy-MM-dd_HH-mm-ss}.png";
}

private void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
    // Copy the raw image data into the target texture
    photoCaptureFrame.UploadImageDataToTexture(targetTexture);

    Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();

    targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
    targetTexture.Apply();

    byte[] bytes = targetTexture.EncodeToPNG();

    string filename = FileName(Convert.ToInt32(targetTexture.width), Convert.ToInt32(targetTexture.height));
    //save to folder under assets
    File.WriteAllBytes(Application.streamingAssetsPath + "/Snapshots/" + filename, bytes);
    Debug.Log("The picture was uploaded");

    // Deactivate the camera
    photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}

private void OnStoppedPhotoMode(PhotoCapture.PhotoCaptureResult result)
{
    // Shutdown the photo capture resource
    VuforiaBehaviour.Instance.gameObject.SetActive(true);
    photoCaptureObject.Dispose();
    photoCaptureObject = null;


}

对不起，如果这被视为与 this 重复。

---

编辑

当我达到这一点时，this 可能会有用。

是不是我根本不需要这三行？

Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();

正如 cmets 中所写，使用这三行与不使用这三行的区别在于保存的照片具有黑色背景 + AR-GUI。上面没有第二行代码的是一张带有 AR-GUI 的照片，但背景是我电脑网络摄像头的实时流。我真的不想看到计算机网络摄像头，而是看到 HoloLens 看到的。

Answer 1

你的三行

Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();

targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();

对我来说没有多大意义。 Texture2D.ReadPixels 用于创建屏幕截图，这样您就可以用屏幕截图覆盖您刚刚从PhotoCapture 收到的纹理？ （同样尺寸不正确，因为相机分辨率很可能！= 屏幕分辨率。）

这也是原因

正如 cmets 中所写，使用这三行与不使用这三行的区别在于保存的照片具有黑色背景 + AR-GUI。

做完之后

photoCaptureFrame.UploadImageDataToTexture(targetTexture);

您已经从targetTexture 中的PhotoCapture 收到了Texture2D。

我想你可能会将它与Texture2D.GetPixels 混淆，后者用于获取给定Texture2D 的像素数据。

---

我想最后从中心裁剪捕获的照片，我想也许这可以通过这个代码行来实现？在 0, 0) 以外的其他像素处开始新的矩形

您真正想要的是从中心裁剪收到的Texture2D，正如您在 cmets 中提到的那样。您可以使用GetPixels(int x, int y, int blockWidth, int blockHeight, int miplevel) 来执行此操作，该GetPixels(int x, int y, int blockWidth, int blockHeight, int miplevel) 用于剪切给定Texture2D 的某个区域

public static Texture2D CropAroundCenter(Texture2D input, Vector2Int newSize)
{
    if(input.width < newSize.x || input.height < newSize.y)
    {
        Debug.LogError("You can't cut out an area of an image which is bigger than the image itself!", this);
        return null;
    }

    // get the pixel coordinate of the center of the input texture
    var center = new Vector2Int(input.width / 2, input.height / 2);

    // Get pixels around center
    // Get Pixels starts width 0,0 in the bottom left corner
    // so as the name says, center.x,center.y would get the pixel in the center
    // we want to start getting pixels from center - half of the newSize 
    //
    // than from starting there we want to read newSize pixels in both dimensions
    var pixels = input.GetPixels(center.x - newSize.x / 2, center.y - newSize.y / 2, newSize.x, newSize.y, 0);

    // Create a new texture with newSize
    var output = new Texture2D(newSize.x, newSize.y);

    output.SetPixels(pixels);
    output.Apply();

    return output;
} 

为了（希望）更好地理解，这是一个说明GetPixels 重载与给定值在这里的作用：

而不是在

中使用它

private void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
    // Copy the raw image data into the target texture
    photoCaptureFrame.UploadImageDataToTexture(targetTexture);

    // for example take only half of the textures width and height
    targetTexture = CropAroundCenter(targetTexture, new Vector2Int(targetTexture.width / 2, targetTexture.height / 2);

    byte[] bytes = targetTexture.EncodeToPNG();

    string filename = FileName(Convert.ToInt32(targetTexture.width), Convert.ToInt32(targetTexture.height));
    //save to folder under assets
    File.WriteAllBytes(Application.streamingAssetsPath + "/Snapshots/" + filename, bytes);
    Debug.Log("The picture was uploaded");

    // Deactivate the camera
    photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}

---

或者您可以将其设置为 extension method 与 static class 类似

public static class Texture2DExtensions
{
    public static void CropAroundCenter(this Texture2D input, Vector2Int newSize)
    {
        if (input.width < newSize.x || input.height < newSize.y)
        {
            Debug.LogError("You can't cut out an area of an image which is bigger than the image itself!");
            return;
        }

        // get the pixel coordinate of the center of the input texture
        var center = new Vector2Int(input.width / 2, input.height / 2);

        // Get pixels around center
        // Get Pixels starts width 0,0 in the bottom left corner
        // so as the name says, center.x,center.y would get the pixel in the center
        // we want to start getting pixels from center - half of the newSize 
        //
        // than from starting there we want to read newSize pixels in both dimensions
        var pixels = input.GetPixels(center.x - newSize.x / 2, center.y - newSize.y / 2, newSize.x, newSize.y, 0);

        // Resize the texture (creating a new one didn't work)
        input.Resize(newSize.x, newSize.y);

        input.SetPixels(pixels);
        input.Apply(true);
    }
}

并像使用它一样使用它

targetTexture.CropAroundCenter(new Vector2Int(targetTexture.width / 2, targetTexture.height / 2));

---

注意：

UploadImageDataToTexture：如果您在 CameraParameters 中指定了 BGRA32 格式，则只能使用此方法。

幸运的是你还是用了它；）

请记住，此操作将发生在主线程上，因此速度很慢。

然而，唯一的选择是CopyRawImageDataIntoBuffer 并在另一个线程或外部生成纹理，所以我想说留在UploadImageDataToTexture 是可以的；）

和

捕获的图像也会在 HoloLens 上显示翻转。您可以使用自定义着色器重新定向图像。

通过翻转它们实际上意味着纹理的Y-Axis 是颠倒的。 X-Axis 是正确的。

要垂直翻转纹理，您可以使用第二种扩展方法：

public static class Texture2DExtensions
{
    public static void CropAroundCenter(){....}

    public static void FlipVertically(this Texture2D texture)
    {
        var pixels = texture.GetPixels();
        var flippedPixels = new Color[pixels.Length];

        // These for loops are for running through each individual pixel and 
        // write them with inverted Y coordinates into the flippedPixels
        for (var x = 0; x < texture.width; x++)
        {
            for (var y = 0; y < texture.height; y++)
            {
                var pixelIndex = x + y * texture.width;
                var flippedIndex = x  + (texture.height - 1 - y) * texture.width;

                flippedPixels[flippedIndex] = pixels[pixelIndex];
            }
        }

        texture.SetPixels(flippedPixels);
        texture.Apply();
    }
}

并像使用它

targetTexture.FlipVertically();

---

结果：（对于这个示例和给定的纹理，我使用了 FlipVertically 并每秒裁剪到一半大小，但对于拍摄的照片应该同样有效。）

图片来源：http://developer.vuforia.com/sites/default/files/sample-apps/targets/imagetargets_targets.pdf

---

更新

到你的按钮问题：

不要使用

if (Input.GetKey("k") || Input.GetKey("k"))

首先，您要检查完全相同的条件两次。而GetKey 会在按键保持按下状态时触发每一帧。而是使用

if (Input.GetKeyDown("k"))

只触发一次。我猜 Vuforia 和 PhotoCapture 存在问题，因为您的原始版本经常触发，并且您可能有一些并发的 PhotoCapture 进程...