使用 Tensorflow 2 在 Android 上进行对象检测答案

【问题标题】：Object Detection on Android using Tensorflow 2使用 Tensorflow 2 在 Android 上进行对象检测
【发布时间】：2020-10-06 10:55:44
【问题描述】：

我正在尝试在 android 上应用自定义对象检测模型。要应用该模型，我使用 /lite/examples/object_detection 下的 tensorflow repository example。出于这个原因，我也在使用我的个人手机（小米红米 Note 8 pro，Android 10）进行测试。该示例完美运行，能够识别不同的对象。但是，当我尝试导入我的自定义模型时，应用程序崩溃反复。要运行我在 build.gradle(:app) 中添加的模型，需要以下依赖项：

implementation('org.tensorflow:tensorflow-lite:0.0.0-nightly') { changing = true }
implementation('org.tensorflow:tensorflow-lite-gpu:0.0.0-nightly') { changing = true }
implementation('org.tensorflow:tensorflow-lite-support:0.0.0-nightly') { changing = true }

以及我已经用我的文件替换了 detect.tflite 和 labelmap.txt。此外，我尝试更改 DetectorActivity.java（TF_OD_API_INPUT_SIZE 和 TF_OD_API_IS_QUANTIZED），因为我使用的模型（ssd_mobilenet_v2_fpnlite _640x640_coco17_tpu-8) 具有不同的输入大小，但它仍然崩溃。有指导Running TF2 Detection API Models on mobile，但我还没有设法实现。我在 Android 上的程序有问题还是模型有问题？谢谢！

我正在添加崩溃的 Logcat 输出：

2020-10-06 19:30:28.163 25354-25354/? E/mples.detectio: Unknown bits set in runtime_flags: 0x8000
2020-10-06 19:30:28.407 25354-25354/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "ro.vendor.df.effect.conflict"
2020-10-06 19:30:28.560 25354-25354/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "camera.aux.packagelist"
2020-10-06 19:30:28.560 25354-25354/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "camera.aux.packagelist"
2020-10-06 19:30:28.560 25354-25354/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "camera.aux.packagelist"
2020-10-06 19:30:28.560 25354-25354/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "camera.aux.packagelist"
2020-10-06 19:30:28.560 25354-25354/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "camera.aux.packagelist"
2020-10-06 19:30:28.560 25354-25354/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "camera.aux.packagelist"
2020-10-06 19:30:28.560 25354-25354/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "camera.aux.packagelist"
2020-10-06 19:30:28.560 25354-25372/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "camera.aux.packagelist"
2020-10-06 19:30:28.561 25354-25372/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "camera.aux.packagelist"
2020-10-06 19:30:28.561 25354-25372/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "camera.aux.packagelist"
2020-10-06 19:30:28.561 25354-25372/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "camera.aux.packagelist"
2020-10-06 19:30:28.626 25354-25354/org.tensorflow.lite.examples.detection E/GraphicExt: Can't load libboost_ext_fwk
2020-10-06 19:30:28.627 25354-25354/org.tensorflow.lite.examples.detection E/GraphicExt: GraphicExtModuleLoader::CreateGraphicExtInstance false
2020-10-06 19:30:28.642 25354-25354/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "vendor.debug.bq.dump"
2020-10-06 19:30:28.642 25354-25354/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "vendor.debug.bq.dump"
2020-10-06 19:30:28.642 25354-25354/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "vendor.debug.bq.dump"
2020-10-06 19:30:28.642 25354-25354/org.tensorflow.lite.examples.detection E/GraphicExt: GraphicExtModuleLoader::CreateGraphicExtInstance false
2020-10-06 19:30:28.657 25354-25354/org.tensorflow.lite.examples.detection E/AndroidRuntime: FATAL EXCEPTION: main
    Process: org.tensorflow.lite.examples.detection, PID: 25354
    java.lang.IllegalStateException: This model does not contain associated files, and is not a Zip file.
        at org.tensorflow.lite.support.metadata.MetadataExtractor.assertZipFile(MetadataExtractor.java:325)
        at org.tensorflow.lite.support.metadata.MetadataExtractor.getAssociatedFile(MetadataExtractor.java:165)
        at org.tensorflow.lite.examples.detection.tflite.TFLiteObjectDetectionAPIModel.create(TFLiteObjectDetectionAPIModel.java:116)
        at org.tensorflow.lite.examples.detection.DetectorActivity.onPreviewSizeChosen(DetectorActivity.java:99)
        at org.tensorflow.lite.examples.detection.CameraActivity$7.onPreviewSizeChosen(CameraActivity.java:446)
        at org.tensorflow.lite.examples.detection.CameraConnectionFragment.setUpCameraOutputs(CameraConnectionFragment.java:357)
        at org.tensorflow.lite.examples.detection.CameraConnectionFragment.openCamera(CameraConnectionFragment.java:362)
        at org.tensorflow.lite.examples.detection.CameraConnectionFragment.access$300(CameraConnectionFragment.java:66)
        at org.tensorflow.lite.examples.detection.CameraConnectionFragment$3.onSurfaceTextureAvailable(CameraConnectionFragment.java:171)
        at android.view.TextureView.getTextureLayer(TextureView.java:406)
        at android.view.TextureView.draw(TextureView.java:349)
        at android.view.View.updateDisplayListIfDirty(View.java:20568)
        at android.view.View.draw(View.java:21448)
        at android.view.ViewGroup.drawChild(ViewGroup.java:4477)
        at android.view.ViewGroup.dispatchDraw(ViewGroup.java:4237)
        at android.view.View.updateDisplayListIfDirty(View.java:20559)
        at android.view.View.draw(View.java:21448)
        at android.view.ViewGroup.drawChild(ViewGroup.java:4477)
        at android.view.ViewGroup.dispatchDraw(ViewGroup.java:4237)
        at android.view.View.updateDisplayListIfDirty(View.java:20559)
        at android.view.View.draw(View.java:21448)
        at android.view.ViewGroup.drawChild(ViewGroup.java:4477)
        at android.view.ViewGroup.dispatchDraw(ViewGroup.java:4237)
        at android.view.View.draw(View.java:21740)
        at android.view.View.updateDisplayListIfDirty(View.java:20568)
        at android.view.View.draw(View.java:21448)
        at android.view.ViewGroup.drawChild(ViewGroup.java:4477)
        at androidx.coordinatorlayout.widget.CoordinatorLayout.drawChild(CoordinatorLayout.java:1246)
        at android.view.ViewGroup.dispatchDraw(ViewGroup.java:4237)
        at android.view.View.draw(View.java:21740)
        at android.view.View.updateDisplayListIfDirty(View.java:20568)
        at android.view.View.draw(View.java:21448)
        at android.view.ViewGroup.drawChild(ViewGroup.java:4477)
        at android.view.ViewGroup.dispatchDraw(ViewGroup.java:4237)
        at android.view.View.updateDisplayListIfDirty(View.java:20559)
        at android.view.View.draw(View.java:21448)
        at android.view.ViewGroup.drawChild(ViewGroup.java:4477)
        at android.view.ViewGroup.dispatchDraw(ViewGroup.java:4237)
        at android.view.View.updateDisplayListIfDirty(View.java:20559)
        at android.view.View.draw(View.java:21448)
        at android.view.ViewGroup.drawChild(ViewGroup.java:4477)
        at android.view.ViewGroup.dispatchDraw(ViewGroup.java:4237)
        at android.view.View.updateDisplayListIfDirty(View.java:20559)
        at android.view.View.draw(View.java:21448)
        at android.view.ViewGroup.drawChild(ViewGroup.java:4477)
        at android.view.ViewGroup.dispatchDraw(ViewGroup.java:4237)
        at android.view.View.updateDisplayListIfDirty(View.java:20559)
        at android.view.View.draw(View.java:21448)
        at android.view.ViewGroup.drawChild(ViewGroup.java:4477)
        at android.view.ViewGroup.dispatchDraw(ViewGroup.java:4237)
        at android.view.View.draw(View.java:21740)
        at com.android.internal.policy.DecorView.draw(DecorView.java:844)
        at android.view.View.updateDisplayListIfDirty(View.java:20568)
        at android.view.ThreadedRenderer.updateViewTreeDisplayList(ThreadedRenderer.java:575)
        at android.view.ThreadedRenderer.updateRootDisplayList(ThreadedRenderer.java:581)
        at android.view.ThreadedRenderer.draw(ThreadedRenderer.java:654)
2020-10-06 19:30:28.658 25354-25354/org.tensorflow.lite.examples.detection E/AndroidRuntime:     at android.view.ViewRootImpl.draw(ViewRootImpl.java:3877)
        at android.view.ViewRootImpl.performDraw(ViewRootImpl.java:3668)
        at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:2988)
        at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:1888)
        at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:8043)
        at android.view.Choreographer$CallbackRecord.run(Choreographer.java:969)
        at android.view.Choreographer.doCallbacks(Choreographer.java:793)
        at android.view.Choreographer.doFrame(Choreographer.java:728)
        at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:954)
        at android.os.Handler.handleCallback(Handler.java:914)
        at android.os.Handler.dispatchMessage(Handler.java:100)
        at android.os.Looper.loop(Looper.java:224)
        at android.app.ActivityThread.main(ActivityThread.java:7551)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:539)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:995)

更新：使用不同的设备（小米红米 Note 4X，Android 7）自定义应用程序可以工作，但无法识别带有边界框的海关对象。因此，对于我来说，Android 10 肯定存在问题，而对于第二种情况，我猜想是 labelmap.txt 文件的问题（如果不是经过训练的模型）。

【问题讨论】：

请分享崩溃的 logcat 输出。
@ShubhamPanchal 我刚刚在我的问题中添加了它。谢谢。

标签： java android android-studio tensorflow object-detection

【解决方案1】：

为了回答我的问题，关于自定义对象检测模型的实现的主要问题是缺少应该附加到模型文件的元数据文件（在我的例子中是 .tflite）。换句话说，一个描述并指定模型的文件、android 应用程序中输入图像的分辨率、您想要使用的标签文件等（因此应用程序期望分辨率为 300x300 而不是 640x640这是我想喂的那个，这就是崩溃的原因）。有关更多信息，您可以查看相关问题：here。

【讨论】：