如何将字符串作为输入并在 ml 引擎上运行

Question

我想在 Google ML Engine 上构建一个中文情绪分析。 我的输入是作为句子的字符串，我还需要做一些字符串处理，例如替换换行符、将字符串拆分为字符并将字符序列填充为固定长度。

这是我的示例代码，我想试试我的想法：

import tensorflow as tf
input_x = tf.placeholder(tf.string, [None])
cleaned_x = tf.regex_replace(input_x, '[\s]+', '')
chars_x = tf.string_split(input_x, delimiter='', skip_empty=True)
paddings = [[0, 0], [0, 1000 - chars_x.dense_shape[0]]]
padded_x = tf.pad([chars_x.values], paddings, 'CONSTANT', constant_values='<UNK>')
sess = tf.Session()
sess.run(tf.global_variables_initializer())
sess.run(tf.local_variables_initializer())

print(sess.run(input_x, feed_dict={input_x:['abcd']}))

但我得到了例外：

(tensorflow) chinshunjus-MBP:Desktop chenjunru$ python test.py 
Traceback (most recent call last):
  File "test.py", line 7, in <module>
    padded_x = tf.pad([chars_x.values], paddings, 'CONSTANT', constant_values='<UNK>')
  File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1922, in pad
    tensor, paddings, constant_values, name=name)
  File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 4363, in pad_v2
    constant_values=constant_values, name=name)
  File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 513, in _apply_op_helper
    raise err
  File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 510, in _apply_op_helper
    preferred_dtype=default_dtype)
  File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1040, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 235, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 214, in constant
    value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 433, in make_tensor_proto
    _AssertCompatible(values, dtype)
  File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 341, in _AssertCompatible
    raise TypeError("List of Tensors when single Tensor expected")
TypeError: List of Tensors when single Tensor expected

我找不到原因，非常感谢您的帮助。

或者如何实现情感分析，输入是字符串数组的情况，谢谢。

[update] 删除 [] 后，也会得到相同的异常。

Traceback (most recent call last):
File "test.py", line 7, in <module>
  padded_x = tf.pad(chars_x.values, paddings, 'CONSTANT', constant_values='<UNK>')
File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1922, in pad
  tensor, paddings, constant_values, name=name)
File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 4363, in pad_v2
  constant_values=constant_values, name=name)
File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 513, in _apply_op_helper
  raise err
File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 510, in _apply_op_helper
  preferred_dtype=default_dtype)
File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1040, in internal_convert_to_tensor
  ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 235, in _constant_tensor_conversion_function
  return constant(v, dtype=dtype, name=name)
File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 214, in constant
  value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 433, in make_tensor_proto
  _AssertCompatible(values, dtype)
File "/Users/chenjunru/Documents/python_workspace/tensorflow/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 341, in _AssertCompatible
  raise TypeError("List of Tensors when single Tensor expected")
TypeError: List of Tensors when single Tensor expected

[update] 更改填充后，得到以下异常：

Traceback (most recent call last):
  File "test.py", line 6, in <module>
    paddings = tf.concat([[[0, 0]], [[0, 1000 - chars_x.dense_shape[0]]]], 0)
  File "/Users/chenjunru/Documents/python_workspace/mlengine27/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 1181, in concat
    return gen_array_ops.concat_v2(values=values, axis=axis, name=name)
  File "/Users/chenjunru/Documents/python_workspace/mlengine27/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 949, in concat_v2
    "ConcatV2", values=values, axis=axis, name=name)
  File "/Users/chenjunru/Documents/python_workspace/mlengine27/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 483, in _apply_op_helper
    raise TypeError("%s that don't all match." % prefix)
TypeError: Tensors in list passed to 'values' of 'ConcatV2' Op have types [int32, int64] that don't all match.

Answer 1

代码中可能还有其他错误，但是堆栈跟踪已经回答了你的问题：

  File "test.py", line 7, in <module>
    padded_x = tf.pad([chars_x.values], paddings, 'CONSTANT', constant_values='<UNK>')

TypeError: List of Tensors when single Tensor expected

也就是说，您将张量列表传递给tf.pad，而它需要一个张量（参见the docs）。此外，TF 将类似地将 paddings 参数视为张量列表而不是二维矩阵。要解决这两个问题：

paddings = tf.concat([
    tf.constant([[0, 0]], dtype=tf.int64),
    [[0, 1000 - chars_x.dense_shape[0]]]
  ], 0)
padded_x = tf.pad(chars_x.values, paddings, 'CONSTANT', constant_values='<UNK>')

请注意，我没有扫描代码以了解更多问题。