OpenGL视频帧拟合

Question

我正在做一个项目，我必须将来自分辨率为 640x480 的相机的数据以纵向模式投射到 4K 屏幕上。

相机是 Kinect V1，但我会切换到分辨率更高的版本 2 (1920x1080)。

我的问题是如何更改要显示的纹理的比例以获得正确的结果。 目前，我已经设法在整个屏幕上显示，但图像的宽度是扁平的。理想的情况是保持比例并在图像的每一侧切割一个 X 宽度。

我正在使用 SDL 和 OpenGL，这里是代码的相关部分：

// window creation
auto window = SDL_CreateWindow("Imagine",
                                   x,
                                   y,
                                   0,
                                   0,
                                   SDL_WINDOW_OPENGL | SDL_WINDOW_SHOWN | SDL_WINDOW_FULLSCREEN_DESKTOP | SDL_WINDOW_ALLOW_HIGHDPI | SDL_WINDOW_BORDERLESS);

// GL initialization and texture creation
void SdlNuitrackRenderHandler::initTexture(int width, int height)
{
    glEnable(GL_TEXTURE_2D);
    glEnableClientState(GL_VERTEX_ARRAY);
    glDisableClientState(GL_COLOR_ARRAY);
    glOrtho(0, _width, _height, 0, -1.0, 1.0);
    glMatrixMode(GL_PROJECTION);
    glPushMatrix();
    glLoadIdentity();

    glGenTextures(1, &_textureID);

    width = power2(width);
    height = power2(height);

    if (_textureBuffer != 0)
        delete[] _textureBuffer;

    _textureBuffer = new uint8_t[width * height * 3];
    memset(_textureBuffer, 0, sizeof(uint8_t) * width * height * 3);

    glBindTexture(GL_TEXTURE_2D, _textureID);
    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, width, height, 0, GL_RGB, GL_UNSIGNED_BYTE, NULL);

    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

    // Set texture coordinates [0, 1] and vertexes position
    {
        _textureCoords[0] = (float) _width / width;
        _textureCoords[1] = (float) _height / height;
        _textureCoords[2] = (float) _width / width;
        _textureCoords[3] = 0.0;
        _textureCoords[4] = 0.0;
        _textureCoords[5] = 0.0;
        _textureCoords[6] = 0.0;
        _textureCoords[7] = (float) _height / height;

        _vertexes[0] = _width;
        _vertexes[1] = _height;
        _vertexes[2] = _width;
        _vertexes[3] = 0.0;
        _vertexes[4] = 0.0;
        _vertexes[5] = 0.0;
        _vertexes[6] = 0.0;
        _vertexes[7] = _height;
    }

// Texture rendering
// Render prepared background texture
void SdlNuitrackRenderHandler::renderTexture()
{
    glClearColor(1, 1, 1, 1);
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    glEnable(GL_TEXTURE_2D);
    glColor4f(1, 1, 1, 1);

    glBindTexture(GL_TEXTURE_2D, _textureID);
    glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, _width, _height, GL_RGB, GL_UNSIGNED_BYTE, _textureBuffer);

    glEnableClientState(GL_VERTEX_ARRAY);
    glEnableClientState(GL_TEXTURE_COORD_ARRAY);

    glVertexPointer(2, GL_FLOAT, 0, _vertexes);
    glTexCoordPointer(2, GL_FLOAT, 0, _textureCoords);

    glDrawArrays(GL_TRIANGLE_FAN, 0, 4);

    glDisableClientState(GL_VERTEX_ARRAY);
    glDisableClientState(GL_TEXTURE_COORD_ARRAY);

    glDisable(GL_TEXTURE_2D);
}

Answer 1

虽然我同意 cmets 中关于这个过时的 OpenGL 代码所写的内容，但这个问题的核心与 OpenGL 无关。您想在另一个具有不同纵横比的矩形内绘制一个具有正确纵横比的矩形。您只需要知道放置顶点的​​位置即可。

通常使用TEXTURE_2D 纹理目标，您希望纹理坐标在两个方向上都是 0-1，除非您打算裁剪输入图像。曾经有一段时间，纹理的宽度和高度必须是 2 的幂。很长一段时间以来都不是这种情况。所以删除这两行：

width = power2(width);
height = power2(height);

所以首先要做的是正确设置：

    _textureCoords[0] = 1.0;
    _textureCoords[1] = 1.0;
    _textureCoords[2] = 1.0;
    _textureCoords[3] = 0.0;
    _textureCoords[4] = 0.0;
    _textureCoords[5] = 0.0;
    _textureCoords[6] = 0.0;
    _textureCoords[7] = 1.0;

（因此，该代码真的很难阅读并且维护起来很麻烦。您应该将纹理坐标（和顶点坐标）设置为 struct 和 x 和y 值，所以它是有道理的。现在并不明显它是 4 组 2D 坐标，即 (max, max), (max, min), (min, min), (min, max)。但我离题了.)

接下来，要计算纹理坐标，您需要知道视频是要缩放以适应宽度还是高度。为此，您可以弄清楚

double widthScaleRatio = displayWidth / imageWidth; // <- using this scale will guarantee the width of the new image is the same as the display's width, but might crop the height
double heightScaleRatio = displayHeight / imageHeight; // <- using this scale will guarantee the height of the new image is the same as the display's height but might crop the width
double  scale = 1.0;
// If scaling by the widthScaleRatio makes the height too big, use the heightScaleRatio
// Otherwise use the widthScaleRatio
if (imageHeight * widthScaleRatio > displayHeight)
{
    scale = heightScaleRatio;
}
else
{
    scale = widthScaleRatio;
}

现在通过scale 缩放宽度和高度：

double newWidth  = imageWidth * scale;
double newHeight = imageHeight * scale;

并基于此设置您的顶点：

    _vertexes[0] = newWidth;
    _vertexes[1] = newHeight;
    _vertexes[2] = newWidth;
    _vertexes[3] = 0.0;
    _vertexes[4] = 0.0;
    _vertexes[5] = 0.0;
    _vertexes[6] = 0.0;
    _vertexes[7] = newHeight;

同样的警告也适用于使这段代码更容易阅读，就像纹理坐标一样。

编辑：这是一个简单的程序来展示它的工作原理：

int main(){
    double  displayWidth    = 2160;
    double  displayHeight   = 4096;
    double  imageWidth  =  640;
    double  imageHeight = 480;
    double widthScaleRatio = displayWidth / imageWidth; // <- using this scale will guarantee the width of the new image is the same as the display's width, but might crop the height
    double heightScaleRatio = displayHeight / imageHeight; // <- using this scale will guarantee the height of the new image is the same as the display's height but might crop the width
    double  scale = 1.0;
    // If scaling by the widthScaleRatio makes the height too big, use the heightScaleRatio
    // Otherwise use the widthScaleRatio
    if (imageHeight * widthScaleRatio > displayHeight)
    {
        scale = heightScaleRatio;
    }
    else
    {
        scale = widthScaleRatio;
    }

    double newWidth  = imageWidth * scale;
    double newHeight = imageHeight * scale;

    std::cout << "New size = (" << newWidth << ", " << newHeight  << ")\n";
}

当我运行它时，我得到：

新尺寸 = (2160, 1620)