如何在 ROS 中使用 OpenCV-Python

Question

在 ROS 环境中使用 yolo 和 opencv-python 我想在ROS中使用yolo和Opencv-python来控制相机并实现物体检测。 现在我已经知道如何在 Windows 中运行 yolo，但是我不知道如何在 ROS 中运行它。 如何将我的代码移植到 ROS 中？

Answer 1

ROS 是一个用于轻松组合不同库的框架，它提供的接口不是定义为头文件，而是定义为“节点”，由“启动文件”（xml 脚本）调用。 这意味着您既希望在视频/相机源上运行 Yolo，又希望它与其他库或代码交互。如果你不这样做，那么你就不需要 ROS。

ROS(v1) 几乎现在在 Ubuntu 上运行得最好。它既可以在本地运行，也可以在 virtualbox 中运行。 ROS2 支持 windows，但如果你有问题，那就是另一个问题了。

要为您的 python 代码创建一个 ROS 节点，首先将其全部放入一个单独的类/模块中； ROS 节点应该只在“真实代码”和通信之间具有接口样板。假设您使用usb_cam_node 获取摄像头供稿，则数据将发布在“主题”<camera_name>/image [sensor_msgs/Image] 上，其中<camera_name> 是usb_cam_node 参数。主题就像所有 ROS 节点之间的全局变量，由Subscriber（带有回调）读取并由Publisher 发布。

然后你必须决定从中发布什么。因为是 yolo，也许你想要每个检测的边界框。有一堆预定义的 ROS 消息（这些是“主题”的（静态和强类型）“类型”）。一个是geometry_msgs/PolygonStamped，它允许您指定一个框的角，并对其进行标记。

这是一些示例代码，取自wiki

# yolo_boxes_node.py
import rospy
from std_msgs.msg import Header
from geometry_msgs.msg import PolygonStamped, Point32
from sensor_msgs.msg import Image
# This is your custom yolo code
import my_yolo as yolo # Assuming a method like as follows:
# yolo.evaluate(img_frame) -> 
#   boxes ([x,y,width,hight] list), confidences (list), classids (list)

# Subscribers
#     img_sub (sensor_msgs/Image): "webcam/image" #Comment: we document a name for the sub, the type, and the default topic for it

# Publishers
#     boxes_pub (geometry_msgs/PolygonStamped): "webcam/yolo/boxes"

# Publishers
boxes_pub = None

# Parameters
frequency = 100.0 # Hz

# Global Variables
img_frame = None
header = None

def img_callback(data): # data of type Image
    global img_frame
    global header
    img_frame = data.data
    header = data.header

def timer_callback(event): # This is to process data at a fixed rate, perhaps different from camera framerate
    # Convert img_frame somehow if needed
    if img_frame is None or boxes_pub is None:
        return
    boxes, confidences, classids = yolo.evaluate(img_frame)
    for b in boxes:
        msg = PolygonStamped()
        msg.header = header # You could use the header differently
        msg.polygon.points.append(Point32(x=b[0],y=b[1]))
        msg.polygon.points.append(Point32(x=b[0]+b[2],y=b[1]))
        msg.polygon.points.append(Point32(x=b[0],y=b[1]+b[3]))
        msg.polygon.points.append(Point32(x=b[0]+b[2],y=b[1]+b[3]))
        boxes_pub.publish(msg)

# In your main function, you subscribe to topics
def yolo_boxes_node():
    # Init ROS
    rospy.init_node('yolo_boxes_node', anonymous=True)

    # Parameters
    if rospy.has_param('~frequency'):
        frequency = rospy.get_param('~frequency')

    # Subscribers
    # Each subscriber has the topic, topic type, AND the callback!
    rospy.Subscriber('webcam/image', Image, img_callback)
    # Rarely/never need to hold onto the object with a variable: 
    #     img_sub = rospy.Subscriber(...)
    rospy.Timer(rospy.Duration(1.0/frequency), timer_callback)

    # Publishers
    boxes_pub = rospy.Publisher('webcam/yolo/boxes', PolygonStamped, queue_size = 100)
    # queue_size increases as buffer for msgs; if you have 1000s of boxes, might need bigger

    # spin() simply keeps python from exiting until this node is stopped
    # This is an infinite loop, the only code that gets ran are callbacks
    rospy.spin()
    # NO CODE GOES AFTER THIS, NONE! USE TIMER CALLBACKS!
    # unless you need to clean up resource allocation, close(), etc when program dies

if __name__ == '__main__':
    yolo_boxes_node()

因此，一个示例 xml 启动文件可能是：

<?xml version="1.0"?>
<!-- my_main_program.launch -->
<launch>
  <!--
    Pub: <camera_name>/image [sensor_msgs/Image]
  -->
  <node name="usb_cam_node" type="usb_cam_node" pkg="usb_cam" output="screen" restart="true">
    <param name="camera_name" value="webcam"/>
    <param name="video_device" value="/dev/video0"/>
  </node>

  <!--
    img_sub: webcam/image [sensor_msgs/Image]
    boxes_pub: webcam/yolo/boxes [geometry_msgs/PolygonStamped]
  -->
  <node name="yolo_boxes_node" type="yolo_boxes_node" pkg="my_pkg" output="screen">
    <param name="frequency" value="30.0"/>
  </node>
</launch>